Contract bidding and awards process audit

Final report

Audit, Evaluation, and Risk Branch
August 2014

Table of Contents

• Executive summary
• Introduction
• Audit objectives
• File review methodology
  • 1.0 Contract bidding and awards practices
  • 1.1 Documentation and records management
  • 2.0 Roles and responsibilities
  • 2.1 Adherence to policies and procedures
  • 3.0 Monitoring
  • 3.1 Adherence to the procurement compliance review directive
  • 3.2 Compliance review process
  • 3.3 Reporting of contracting activities
• Conclusion
• Acknowledgement
• Appendix A - Breakdown of contract files reviewed
• Appendix B - Procurement chart

Executive summary

Background:

A commitment to fairness, openness, and transparency in procurement is a requirement set out by the Government of Canada in the Financial Administration Act. Section 61 of the Canada Revenue Agency Act provides the Canada Revenue Agency (CRA) with full procurement authority for goods and services, excluding legal services, and grants the CRA authority to enter into contracts with governments and public or private organizations (construction services are also outside CRA's procurement responsibility).

Within the CRA, the Commissioner and Chief Executive Officer is responsible for executive oversight of the procurement regime. Accountability and authority for policies and services related to procurement, including contracting, asset and inventory management or disposal resides with the Administration Directorate (AD) of the Finance and Administration Branch (FAB). The AD is responsible for ensuring the fair, open, equitable, transparent, and cost effective acquisition of goods and services to support the achievement of the Agency's business goals. The AD is also responsible for providing a quality assurance and compliance oversight function for the CRA's procurement activities.

Within the CRA, contracts are categorized as competitive and non-competitive. A competitive contract requires the solicitation of bids which provides all suppliers an equal opportunity for access to CRA business and to compete for CRA procurements. A non-competitive contract is one that is entered into without soliciting bids and must meet one of the following criteria: pressing emergency, low dollar value, only source, or not in public interest.

In fiscal year 2012-2013, the CRA reported a total of 717 new contracts totalling $61M. This total includes both competitive and non-competitive contracts.

Objectives:

The objective of this audit was to provide assurance that the internal controls in the CRA bidding and awards process are in place and functioning as intended.

The audit focused on assessing compliance with the procurement policies and procedures at each of the following three stages of the procurement cycle: requirements definition, contract bid solicitation, and contract award.

This audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing.

Examination activities were conducted from November 2013 to May 2014 and included file and documentation reviews as well as interviews with key internal stakeholders involved in procurement activities.

This internal audit of the Contract Bidding and Awards Process is the third in a series of audits to be conducted as part of the Cyclical Audit Plan for Procurement.

File Review Methodology:

Internal Audit (IA) reviewed a total of 144 contract files that were initiated during the 2012-2013 fiscal year. As part of the file review two different sampling procedures were used to select the files to be examined:

• convenience sampling was used to identify and select the three largest dollar value contracts; and
• random selection was used to obtain the remaining 141 files in order to obtain a statically valid, representative sample, comprised of 41 competitive and 100 non-competitive contracts.

A pattern began to emerge after post review enquiries were obtained where it was found that the “solicitation procedure” was coded inaccurately across both types of files (see Appendix A for details). As a result, it is improbable that the samples drawn are representative and the frequencies and other quantitative results cannot be generalized to the larger population. In spite of this, a decision was made to continue with the audit given the work that had already been completed and the value that could still be provided to the Agency.

Conclusion:

Adherence to the policies and procedures throughout the procurement cycle is critical to ensure that the CRA meets its commitment to fairness, openness, and transparency. While this audit was not designed to specifically uncover integrity lapses auditors were alert to this as a potential risk and exercised due care in the completion of audit testing. No cases of integrity lapses were identified in the sample of 144 contracts reviewed as part of this audit. For the most part, for the files reviewed, the procurement transactions are in compliance with established policies, processes and procedures.

The overarching delegation of procurement authority was respected for the contracts examined in both the non-competitive and competitive sample file reviews. Additionally, for the competitive contracts examined that included a bidding stage, results indicated these contracts had complete and clear bid evaluation criteria and selection methodology documented on file as required by the procurement policy instruments. Throughout the audit, IA has identified opportunities for improved internal controls to promote adherence to procurement policy instruments, completeness of supporting documentation and data accuracy. The following recommendations were made to FAB:

• implement controls to enforce adherence to procurement policy instruments;
• fully document requirements under the various procurement policy instruments;
• strengthen documentation controls to support the reviewing of contracting files;
• update procurement policy instruments for appropriateness and relevancy; and 
• enforce a consistent records management approach for documentation storage to  provide a seamless audit trail.

As a result of the audit findings, the following recommendations have been made to FAB to improve the accuracy and reliability of monitoring and reporting of procurement transactions so that they can be used to support informed decision making:

• education to procurement staff regarding the importance of data accuracy;
• development of comprehensive guidelines to strengthen the monitoring of procurement transactions; and
• strengthen monitoring of accuracy and reliability of key elements of procurement transactions captured for reporting purposes.

Action Plans:

The AD of FAB agreed with the recommendations in this report and has developed action plans to address the audit findings.

The AD plans to revise their file documentation checklist to fully document completion of procurement requirements under the various policy instruments and to ensure the completeness of their procurement files. Peer reviewers/approval authorities will monitor adherence to the updated checklist. The AD will also, update its policy instruments such as the Contracts Directive to address less extensive file documentation requirements for low risk and low value procurement files and will continue to reiterate the mandatory requirement concerning document storage to all staff.

The AD will continue to educate staff on the importance of complete and accurate procurement data and will utilize existing tools to capture the details of all files being processed, regardless of value. These tools will be monitored by managers and assistant directors to ensure the accuracy of the information and will be used to confirm if and when procurement files were submitted for a compliance review. In order to strengthen the monitoring of procurement transactions, AD is working to develop Compliance Review Checklists to ensure consistency in compliance reviews conducted and strengthen the overall monitoring of procurement transactions.

IA has reviewed management's action plans and concluded that they are adequate and reasonable to address the recommendations. To ensure the complete implementation of these plans, IA plans to conduct a limited follow-up audit in 2015-2016.

Introduction

A commitment to fairness, openness, and transparency in procurement is a requirement set out by the Government of Canada in the Financial Administration Act. Section 61 of the Canada Revenue Agency Act provides the Canada Revenue Agency (CRA) with full procurement authority for goods and services, excluding legal services ^{Footnote 1} , and grants the CRA authority to enter into contracts with governments and public or private organizations (construction services ^{Footnote 2} are also outside of CRA's procurement responsibility). Furthermore, under subsection 30(1) of the CRA Act, the CRA has authority over all matters relating to general administrative policy in the Agency, including policies with respect to procurement and contracting.

Within the CRA, the Commissioner and Chief Executive Officer is responsible for executive oversight of the CRA procurement regime. Accountability and authority for policies and services related to procurement, including contracting, asset and inventory management or disposal resides with the Administration Directorate (AD) of the Finance and Administration Branch (FAB). The AD is responsible for ensuring the fair, open, equitable, transparent and cost effective acquisition of goods and services to support the achievement of the Agency's business goals. The AD is also responsible for providing a quality assurance and compliance oversight function for the CRA's procurement activities.

The AD has the following four divisions:

• Business and Management Services
• Contracting
• Agency Logistics and Administrative Services
• Policy and Program Development

Within the Contracting Division there are four sections (Goods and Tax Programs Support, Information Technology, Services and Printing, and Client Support) responsible for directing the development and implementation of strategies, plans, and procedures for the acquisition of goods and services. In addition, the National Centre for Regional Contracting located in Quebec is responsible for most low dollar value regional procurements (i.e. under the trade agreement thresholds), and include some headquarters' procurements.

The Compliance and Program Review Section (CPRS) in the Business and Management Services Division provides assurance that the overall contracting function is being carried out effectively and in accordance with established procurement policies and procedures. This involves verification, compliance audits and evaluation of the various factors that may affect the integrity of the contracting process.

The procurement of goods and services within the CRA involves a broad range of planning and management activities comprising the following five stages:

Graph 1: Procurement Process

The activities for each stage of the procurement process are outlined below:

1. Requirements definition
   • Requirements are identified and defined by the client^{Footnote 3} in sufficient detail to facilitate the timely acquisition of the goods and/or services required.
2. Contract bid solicitation
   • Bids are solicited from potential suppliers and are assessed against pre-established evaluation criteria.
3. Contract award
   • A contract is awarded to the successful supplier on the basis of a pre-established selection methodology.
4. Contract administration
   • This includes the activities that follow contract award and continue through to contract completion and final payment.
5. Post contract
   • The review and close-out of the procurement file is undertaken by the AD upon completion of the contract.

Within the CRA, contracts are categorized as competitive and non-competitive. A competitive contract requires the solicitation of bids. This requirement provides all suppliers, qualified firms, and individuals an equal opportunity for access to CRA business and to compete for CRA procurements.

A non-competitive contract is one that is entered into without soliciting bids and must meet one of the following criteria:

• Pressing emergency – The need is one for which a delay would be injurious to the public interest.
• Low dollar value – The estimated expenditure of the required goods/services (inclusive of all applicable taxes) does not exceed $25,000.
• Only source – Only one person or firm is capable of performing the contract.
• Not in public interest – The nature of the work is such that it would not be in the public interest to follow a competitive process.

In fiscal year 2012-2013, the CRA reported a total of 717 new contracts totalling $61M. This total includes both competitive and non-competitive contracts. Of these 717 contracts, Internal Audit (IA) reviewed 144 contracts.  Appendix A provides a breakdown of the contracts reviewed by IA.

In October 2010, a Cyclical Audit Plan for Procurement (Cyclical Plan)^{Footnote 4} was developed. This Cyclical Plan outlines a series of audits to be conducted over a number of years to provide management with assurance that internal controls are in place and working as intended on key activities throughout the various phases of the procurement process.

The Contract Bidding and Awards Process is the third audit to be conducted as part of the Cyclical Plan.

Appendix B provides detailed information regarding procurement methods used in the CRA.

Audit objectives

The objective of this audit was to provide assurance that the internal controls in the CRA bidding and awards process are in place and functioning as intended.

The audit focused on assessing compliance with the procurement policies and procedures at each of the following three stages of the procurement process (refer to the stages as shown in Graph 1):

• Stage 1:  Requirements definition;
• Stage 2:  Contract bid solicitation; and
• Stage 3:  Contract award.

Examination activities were conducted from November 2013 to May 2014 and included file and documentation reviews as well as interviews with key internal stakeholders involved in procurement activities.

This audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing.

The remaining two stages of the procurement process, the contract administration (stage 4) and post contract (stage 5), are to be examined in future audits as part of the Cyclical Plan.

File review methodology

IA reviewed a total of 144 contract files that were initiated during the 2012-2013 fiscal year. Two different sampling procedures were used to take into account both the dollar value and the solicitation procedure, or contracting method employed (competitive and non-competitive):

• convenience sampling was used to identify and select the three largest dollar value contracts. These 3 contracts accounted for $49.4M, or 78%, of all new Agency contracts within the scope of the audit.
• random selection was used to obtain the remaining 141 files in order to obtain a statistically valid, representative sample^{Footnote 5}, comprised of 41 competitive and 100 non-competitive contracts for the fiscal year 2012-2013.  A breakdown of the dollar value of these files is as follows:

A pattern began to emerge after post review enquiries were obtained where it was found that the “solicitation procedure” was coded inaccurately across both types of files (see Appendix A for details). As a result, it is improbable that the samples drawn are representative and the frequencies and other quantitative results cannot be generalized to the larger population. In spite of this, a decision was made to continue with the audit given the work that had already been completed and the value that could still be provided to the Agency.

To address the sampling concern in the random sample, a qualitative approach has been adopted to reporting the audit findings. Results are reported in quartiles and the following terms appear throughout the report to capture the approximate range of the results:

FINDINGS, RECOMMENDATIONS, AND ACTION PLANS

Line of enquiry 1 – To determine if contract bidding and awards activities are in compliance with procurement policies and procedures.

As stated in the Procurement Planning and Administration Procedures, solicitation instruments for competitive contracts should be clear, justifiable, objective, and measureable. Contract solicitation instruments (e.g. request for quotes, request for proposals) are expected to contain clearly defined conditions that allow for the fair treatment of all potential suppliers.

While this audit was not designed to specifically uncover integrity lapses auditors were alert to this as a potential risk and exercised due care in the completion of audit testing. No cases of integrity lapses were identified in the sample of 144 contracts reviewed as part of this audit.

1.0 Contract bidding and awards practices

Contract bid solicitation instrument

For the CRA to meet the elements of fair, open and transparent^{Footnote 6}, the bid solicitation instrument should be complete, clear, include bid evaluation criteria (e.g. mandatory criteria), and selection methodology.

Mandatory bid evaluation criteria are the obligatory requirements that must be met by a bidder in order for their bid to be further considered in the bid evaluation process. CRA procurement specialists are expected to work with their clients to ensure that the solicitation instrument clearly conveys to prospective bidders that failure to meet any of the mandatory criteria specified in the solicitation instrument will render their bid non-compliant and that it will be given no further consideration.

Results from file review indicated that a large majority of the solicitation instruments had clearly defined mandatory criteria that were separately identified. In addition, as required, the bid solicitation instruments indicated that failure to meet any of the mandatory criteria would render the bid non-compliant with no further consideration.

Selection methodology refers to how the winning bid will be determined based on the evaluation criteria and the process set out in the bid solicitation instrument. The selection methodology (e.g. lowest-priced compliant proposal) to be used must be clearly identified in the bid solicitation. The contractor selection methodology was clearly defined in the large majority of the solicitation instruments reviewed.

Evaluation of bids

The procurement specialist must review the bid to determine its completeness and level of compliance with the mandatory criteria set out in the solicitation instrument. The procurement specialist must also evaluate compliance with those mandatory criteria that are of a financial, contractual, or non-technical nature.

File review results revealed that for a few competitive contracts, documentation was not on file to confirm that the contract bids were evaluated by the procurement specialist against both the financial and non-financial aspects of the mandatory criteria. See Appendix A for breakdown of files reviewed.

Documentation was not always on file to indicate that procurement specialists evaluated the non-financial aspects of the mandatory criteria. In one contract, evidence was not on file to attest that the successful bidder was compliant to the non-financial portion of the mandatory criteria.

Without a sound evaluation of all aspects of the mandatory criteria, the Agency is at risk for contracts to be awarded to a non-compliant bidder. Incomplete documentation prevents the Agency from demonstrating that contracts are awarded in a fair, open, and transparent manner as expected by vendors and the general public. Also, incomplete documentation may affect the Agency's ability to appropriately respond to legal challenges and may tarnish the reputation of the Agency.

Contract award

The Procurement Planning and Administration Procedures state that the competitive approach should be used whenever possible, practical, or cost effective to do so.  In instances when the competitive approach is not used, a non-competitive contract may be awarded to a pre-selected contractor only if it meets one of the following exceptions:

• pressing emergency;
• low dollar value (below $25,000);
• only source; and
• not in public interest to competition.

Results from review of non-competitive contracts (92 out of 100 reviewed were below $25,000) revealed that the large majority of files met one of the allowable exceptions.

Recommendation

FAB should provide training and guidance on a timely basis to all procurement specialists on the bid evaluation aspect of the procurement process.

Action plan
It has been determined that a couple of procurement specialists, in one area were unfamiliar with the requirement to maintain clear file documentation on the bid evaluation criteria. To address this matter, AD has provided training to those individuals.

AD will revise the file documentation checklist to include confirmation by the procurement specialist that evaluation of all bid criteria has been completed. Once revised, the checklist will be disseminated to all procurement specialists.
Completion date:  November 2014

1.1 Documentation and records management

Documentation should be complete and consistent to ensure adequate records management in terms of providing evidence and ensuring that an audit trail is available to demonstrate adherence to all aspects of the procurement process and procedures. It is important that information is on file to facilitate and support informed decision-making. Lack of proper documentation and an inadequate audit trail could potentially impact the Agency's ability to be successful, particularly with respect to legal challenges from bidders and vendors.

For non-competitive contracts, the Contracts Directive stipulates that the procurement file must include:

• which exception to competition applies;
• the rationale for selecting the recipient of the contract;
• the limited tendering reason used where trade agreement requirements apply;
• how the value for money determination was made in the absence of a call for bids; and
• for procurements with a value greater than $25,000, a completed and signed copy of the Source Justification and Limited Tendering Certification document.

Results from review of non-competitive contracts indicate that complete information was not consistently maintained on the procurement file as required by the Contracts Directive to demonstrate adherence to all steps in contracting processes and compliance with associated requirements.

From files reviewed, documented rationale for selecting the recipient of the contract was generally not evident. Responses to follow up enquiry by IA indicated that procurement specialists may not be fully familiar with the documentation requirements, particularly for low dollar value contracts (i.e. less than $25,000). Documented rationale must be on file to support decisions with respect to vendor selection for such contracts.

For procurements valued greater than $10,000, documentation was not evident in a few files to support decisions on value for money (e.g. price support).

As discussed previously under the Contract Award section, results from review of non-competitive contracts revealed that the large majority of files met one of the allowable exceptions to competition.

Non-competitive contracts greater than $25,000

For non-competitive contracts greater than $25,000 and subject to trade agreements, file review results indicate that the limited tendering rationale was documented in the large majority of all applicable files.

In the majority of non-competitive files with a value over $25,000, a copy of the Source Justification and Limited Tendering Certification was not completed and/or appropriately signed. In the Contracts Directive, the requirement is for the certification to be completed and signed. Under the directive, the accountability level of the signature is not specified. However, the certification form requires a director level signature.

Non-competitive Information Technology (IT) contracts under $5,000

Results from file review of IT contracts under $5,000 identified the following documentation deficiencies:

• Documentation to support the exception to competition was not evident in some files.
• In a large majority of the files reviewed, a documented rationale was not on file to support the selection of the vendor.

The IT contracting section within AD has internal procedures for low dollar value procurement which are considered low risk. However, these procedures are not consistent with the Contracts Directive. The documentation requirements in the internal procedures are less extensive than those in the Contracts Directive in order to reflect the low risk nature of such procurement.

Receipt date and time for bids

In recognition of the requirement to treat bidders fairly and without bias, it is important to ensure that the bid handling procedures used for all bids submitted in response to a CRA solicitation provide for an accurate record of the date and time of receipt.^{Footnote 7} Documentation should be on file to confirm the received dates and times of proposals submitted in response to bid solicitations.

For one of the high value contracts posted for tender, the receipt date and time stamp were not evident in any of the bidder files submitted for consideration. Without an original record of the receipt date and time, the CRA is at increased risk of eligibility challenges from bidders and negatively impacts its ability to defend the eligibility of such bids.

Recommendation

FAB should strengthen review controls of contracting files for compliance to the Contracts Directive. ie. contracting processes and associated procedures.   

Action plan
The AD will update the file documentation checklist for low dollar, low risk procurements used by procurement specialists to confirm the completeness of the procurement files. Completion date: November 2014

CPRS will confirm the completion of the updated checklist when reviewing procurement files. In addition, peer reviewers / approval authorities will also monitor adherence to the updated checklist. Completion date: March 2015

Recommendation

FAB should update the Contracts Directive to reflect the documentation requirements for low dollar value IT contracts and to ensure consistency between the internal procedures and the Directive.

Action plan
AD will ensure that there is no inconsistency between its policy instrument(s) and internal procedures in terms of the requirement for less extensive file documentation in regard to its low risk and low value procurement files. The Contracts Directive and/or the Procurement Planning and Administration Procedures will be revised to include file documentation requirements for low risk and low value procurement files, which will be disseminated to all procurement specialists. Completion date: March 2015

Statement of Work (SOW) review

As per the Procurement Planning and Administration Procedures, the preparation of the SOW is the responsibility of the client to clearly describe the work to be carried out, the objectives to be attained, the deliverables to be furnished, and the time frame for completion of the work. It is the responsibility of the procurement specialist to review and challenge the client's procurement request and the documentation provided in support of that request. Identification of potential employer-employee relationships is particularly important when contracting for services.

For both competitive and non-competitive contracts reviewed, although not required, evidence was not consistently on file to confirm that the procurement specialist had reviewed and validated the SOW. Comments from an interview with CRA Legal Services indicated that the SOW is a critical document within the procurement process.

The risk of disputes and associated financial costs, including legal challenges, is increased when SOW information is not clearly expressed and documented. The Agency may incur expenses with minimal value for money as the product and services may not be as expected or value-added if the SOW information is not adequate. Comments from interviews with the CPRS within AD indicated that some of the most common errors were found in the SOW (e.g. not sufficiently comprehensive or too prescriptive).

Employer-employee relationship

Employment status directly affects a person's entitlement to employment insurance (EI) benefits under the Employment Insurance Act. It can also have an impact on how a worker is treated under other legislation such as the Canada Pension Plan and the Income Tax Act. 

The facts of the working relationship as a whole determine the employment status. If the worker is an employee (employer-employee relationship), the payer is considered an employer. Employers are responsible for deducting Canada Pension Plan (CPP) contributions, EI premiums, and income tax from remuneration or other amounts they pay to their employees. They have to remit these deductions along with their share of CPP contributions and EI premiums, to the CRA. An employer who fails to deduct the required CPP contributions or EI premiums has to pay both the employer's share and the employee's share of any contributions and premiums owing, plus penalties and interest.^{Footnote 8}

In accordance with the Procurement Planning and Administration Procedures, the procurement specialists are expected to review the contents of the SOW for conditions that could lead to the establishment of an employer-employee relationship.

A review of non-competitive contracts identified only a few contracts where it could be concluded that the procurement specialist did in fact review whether or not the contract would result in the establishment of an employer-employee relationship.

The procedures do not require documented evidence that a procurement specialist has considered the issue of employer-employee relationship in their review of the SOW. This factor limits the ability of the audit team to assess the extent the procurement specialists are fulfilling their review role with respect to the SOW.

Recommendation

As a best practice, FAB should require procurement specialists to fully document their review of client prepared mandatory documents and their consideration of the employer-employee relationship.

Action plan
While procurement specialists work to protect the CRA from employer-employee relationships by reviewing the wording contained in its contract documents (e.g. SOW), the most critical factor in determining if an employer-employee relationship exists is the actual day-to-day work situation that is set by the project authority. There is a link on the Procurement Place website under the Procurement Clients Corner which provides extensive information on employer-employee relationships. Further, in June 2014 a document titled “Tips for Managers to Avoid Employer-Employee Relationships in the Contracting of Services” was added to the same site. Completion date: Completed

AD will update their existing file documentation checklist used for confirming the completeness of the files, to include confirmation that the SOW has been reviewed and that the potential for employer-employee relationships has been considered (e.g. a statement from the procurement specialist on file saying “I have considered Employer-Employee relationships”). Completion date: November 2014

The Contracting Authority will provide to the project authority a document for their signature, acknowledging that they have read and understood what they must do (or not do) to avoid creating an employer-employee relationship. The signed document (or e-mail acknowledgement) will be retained on the procurement file or Synergy. Completion date: November 2014

Audit Trail

During file review, the audit trail was difficult to follow as records management practices (including storage) varied among procurement specialists. Hard copy and Synergy (web-based procurement and ordering tool) are used interchangeably by procurement specialists to store documentation. Emails are stored on network drives.

The use of Synergy as the primary storage for documentation of procurement files has not been enforced. Comments from interviews with procurement specialists indicated the use of Synergy for records storage varied.

The lack of a common records management approach for documentation storage increases the risk of incomplete documentation for a seamless audit trail.

Recommendation

As a best practice, FAB should develop procedures that enforce the use of Synergy for all contract management processes and review existing controls in place to improve document storage.

Action plan
AD will send a communication to all procurement specialists to advise that Synergy is mandatory for the storage of procurement file documentation to the extent possible. All documentation not in Synergy must be either in the Corporate Administrative System (CAS) or on the procurement file. Completion date: September 2014.

AD will update the Procurement Planning and Administration Procedures to reflect that Synergy is to be used to the extent possible; the remaining documentation must be in CAS or on the file. Appropriate cross references are to be used to maintain a seamless audit trail. Completion date: March 2015

In accordance with the Procurement Compliance Review Directive, a compliance review will examine the accuracy of critical elements in CAS and Synergy, as well as current and complete procurement file documentation. However, because the vast majority of files audited were low risk and low value, most were not subject to a mandatory procurement compliance review.

2.0 Roles and responsibilities

Roles and responsibilities of key internal stakeholders involved in contract bidding and awards activities are defined and documented in the policies and procedures of the Finance and Administration Manual (FAM).

2.1 Adherence to policies and procedures

The overarching delegation of procurement authority^{Footnote 9} was respected for the contracts examined in both the non-competitive and competitive sample file reviews.

As outlined below, procurement specialists do not consistently perform the duties required by their roles and responsibilities as defined in the Procurement, Planning and Administration Procedures.

Coding of solicitation procedure (i.e. competitive, non-competitive)

A comparative review was performed on the documentation in the procurement files with procurement information for the fiscal year 2012-2013 provided by the AD. Results indicated a lack of accuracy in the classification and identification of select aspects of procurement transactions for file review purposes.

Results from the file review include the following coding errors:

• From the initial statistically valid representative sample of 41 competitive files, IA found 14 errors in the classification of the solicitation procedure (i.e. non-competitively awarded contracts identified as being awarded on a competitive basis).
• From the initial statistically valid representative sample of 100 non-competitive files, IA found 6 errors in the classification of the solicitation procedure. IA found a few cases where goods (e.g. printers and scanners) were inappropriately classified as generating intellectual property.
• A few after the fact (ATF)^{Footnote 10} contracts were a result of coding errors within CAS and should not have been classified as ATF. Additionally, a few contracts that should have been classified as ATF were not identified as such.

Monthly review of procurement transactions using the validation tool is an evaluation process performed by the procurement specialists on the accuracy of their own work. The purpose of the monthly review is to validate the accuracy of select procurement elements that are used for reporting and/or disclosure purposes.

Users of this information, including Agency senior management and the Board of Management, may not be able to rely on the completeness and accuracy of reported data to support informed decision-making. Reliable and accurate information may not be available for analysis to identify trends and patterns; support monitoring and quality review (QR) activities.

Recommendation

FAB should educate procurement specialists in the importance of coding accuracy and completeness and perform regular monitoring on these aspects.

Action plan
AD will reiterate the importance of complete and accurate coding to procurement specialists.

The following steps have been or will be taken to strengthen coding accuracy:

• In quarter one of fiscal year 2013-2014, AD reviewed coding related to intellectual property ownership and, as a result, updated the data for the 2013 Procurement Activity Report (annual procurement report shared with Public Works and Government Services Canada) (PWGSC) as well as in CAS. Completion date: Completed
• As the Intellectual Property (IP) Ownership Directive is undergoing a 5-year review, AD took the opportunity to simplify the IP coding instructions for procurement specialists. Completion date: December 2014
• AD will include coding accuracy as part of each procurement specialist's performance expectations that will form part of their annual assessment. Completion date: November 2014
• In April 2014, the CPRS developed and published a CAS Coding Reference Tool (which provides a summary of all coding items in CAS as well as an explanation of each and what the correct code under each category should be). Completion date: Completed
• AD is building a central repository of all CAS related information/instructions related to procurement and coding. Completion date: December 2014

After the Fact (ATF) contracts

The CRA's Procurement Policy and its supporting Contracts Directive state the competitive approach should be used whenever feasible to support the Agency's commitment to an open, fair, and transparent procurement process.

In situations where goods or services are provided without a legally binding contractual document, these are known as ATF contracting. In these circumstances, the CRA is exposed and becomes liable for actions such as lawsuits that can result from accidents, work-related injuries, or damages to third parties. Any direction given to a supplier in the absence of a legally binding contractual document may create a contractual arrangement in law and a potential liability for the CRA. According to the After the Fact Contracting Situations Directive, an ATF contract occurs when a request is made to a supplier to deliver goods or to provide services in the absence of a legally binding contractual document. The objective of the directive is to eliminate ATF contracting situations.^{Footnote 11}

Based on the National Trend Analysis and Review of After the Fact Contracting Situations Fiscal Year 2012-2013, an ATF contracting situation occurs if any of the following conditions are met:

• goods or services are acquired by someone with insufficient or no delegated procurement authority;
• goods or services are requested and/or delivered without a legally binding purchasing document (written contract or purchase order); or
• goods or services are requested and/or delivered after a contract has expired or prior to an amendment being issued to an existing contract, e.g. beyond the existing dollar amount or scope; or
• a partial delivery of goods or services is requested and/or received without a legally binding purchasing document – i.e. future delivery(ies) are anticipated to complete the requirement.^{Footnote 12}

The audit team found a few instances in the non-competitive sample review where the client went directly to a vendor to request services and discuss contractual matters prior to involving the Contracting Division. The definition of an ATF is not consistent between the directive and the National Trend Analysis and Review (NTAR). These instances would not be ATFs under the directive; but, they would be ATFs under the definition in the NTAR.

Recommendation

FAB should clarify in the After the Fact Contracting Situations Directive the definition of ATFs to improve consistency in identifying and coding.

Action plan
ATFs have dramatically declined over the past five years, with an expectation that their occurrence will continue to decline going forward (down by approximately 72% from fiscal year 2008-2009).

AD will update the ATF Contracting Situations Directive to further clarify the definition of what constitutes an ATF. Completion date: December 2014

AD will update the ATF Contracting Situations Directive to define what constitutes “unpreventable” ATFs – i.e. a small percentage of contracts for software maintenance and support services where suppliers are causing the delays in awarding the contract(s). Notwithstanding, for transparency purposes, AD will include these “unpreventable” ATFs in their semi-annual ATF dashboard as well as their annual ATF trend analysis. Completion date: December 2014

Recommendation

FAB should educate clients and procurement staff on the importance of having valid contracts in place to request and receive goods and services, in addition to the consequences of invalid contracts.

Action plan
The Procurement Clients Corner on InfoZone contains detailed information on the roles and responsibilities of clients related to procurement. FAB will reinforce and further promote the importance of involving the Contracting Division as soon as a client identifies a procurement requirement and promote the use of the Procurement Clients Corner. Completion date: December 2014

Line of enquiry 2 – To determine if monitoring and quality review processes are in place and functioning as intended for contract bidding and awards activities.

3.0 Monitoring

Monitoring procedures and processes should be in place to enable management to assess progress towards the achievement of established goals and objectives and to meet quality requirements.

The CPRS provides assurance to the overall contracting function and ensures it is acting in accordance with established procurement policies and procedures. CPRS provides numerous procurement oversight functions, primarily in the form of mandatory and targeted compliance reviews.^{Footnote 13}

3.1 Adherence to the procurement compliance review directive

Mandatory review

The Procurement Compliance Review Directive (PCRD) provides criteria for the types of procurement that are subject to mandatory reviews by CPRS within each stage of the procurement process. Mandatory reviews are conducted at the pre-solicitation, pre-award and post-award stages. Pre-solicitation is prior to obtaining approval to post a solicitation. Pre-award is after bid evaluation and/or negotiations but prior to contract award. CPRS reviews procurements in terms of their compliance with procurement policies, directives, guidelines, and procedures.^{Footnote 14}

The requirements of the PCRD do not always accurately represent the procurement process. According to the mandatory review criteria stated in the PCRD, “proposed sole-source contracts exceeding $25K…”^{Footnote 15} are subject to a mandatory pre-solicitation compliance review. However, there is no pre-solicitation stage in the non-competitive process for contracts valued greater than $25,000.

Quality review activities are not consistently conducted when the files meet the criteria set out in the PCRD. A few of the procurement files reviewed that met the pre-award review criteria of the PCRD were not sent to CPRS for mandatory review. CPRS relies on procurement specialists to submit their own contracting files for review. The principle of segregation of duties is not respected when a monitoring control is not in place to ensure that the files are appropriately identified and actually submitted for review.  A lack of monitoring controls to ensure that files subject to a mandatory compliance review are actually submitted for review could potentially result in the issuance of procurements that are not in line with established procurement policies and procedures, thereby hindering CRA's ability to ensure that all procurement activities are carried out in an open, fair and transparent manner. 

During the examination phase of this audit, CPRS finalized and provided the Procurement Mandatory Compliance Review Guide to procurement specialists as a supplement to the PCRD. The Guide is intended to increase the understanding of the purpose and requirements for the mandatory review of certain files.

Recommendation

FAB should review and update the mandatory criteria in the Procurement Compliance Review Directive for appropriateness and relevancy.

Action plan
AD will revise the Procurement Compliance Review Directive to state that all sole-source contracts over $25K are subject to a mandatory pre-award, rather than a pre-solicitation review. Completion date: March 2015

Recommendation

FAB should clarify and enforce adherence to the files requiring mandatory review in accordance with the Procurement Compliance Review Directive.

Action plan
Team Leaders (Supply Project Managers) as well as Assistant Directors of the Contracting Division will monitor the procurement transactions to ensure that mandatory compliance reviews are being conducted. Completion date: November 2014

Recommendation

FAB should implement controls to respect the principle of segregation of duties.

Action plan
The Contracting Division will utilize existing tools to capture details of all files being processed, regardless of value. One element to be included is whether a mandatory compliance review is required. Team Leaders (Supply Project Managers) as well as Assistant Directors will monitor to ensure the accuracy of the information and will follow up with the procurement authorities to confirm if and when a file was submitted for a compliance review, as applicable. On a monthly basis, CPRS will confirm they have received the applicable files for review. Completion date: November 2014

3.2 Compliance review process

Support tools

Mandatory compliance reviews are performed by CPRS reviewers on a subjective basis, primarily relying on their own knowledge of procurement and individual judgement.  Specific review requirements have not been developed and communicated. Consequently, compliance or quality reviews are not conducted consistently among reviewers.

Comments from interviews with CPRS reviewers indicated that existing tools such as the web-based Compliance Review Tool is used to capture results of all reviews. However, without the establishment of specific review requirements, relevant and reliable information is not captured and documented to support trend analysis.

Recommendation

FAB should develop comprehensive guidelines to strengthen the review process for consistency and to capture relevant and reliable information that is timely and useful to identify trends and patterns.

Action plan
In May 2014, CPRS developed and published a Procurement Mandatory Compliance Review Guide, providing a more detailed overview of the CPRS mandate as well as providing contracting officers with a better understanding of what the mandatory compliance review process entails.

CPRS is also working on developing Compliance Review Checklists that will better ensure consistency in reviews and allow for the better identification of trends and patterns. Completion date: March 2015

Follow up and trend analysis

Currently, the CPRS mandate and process do not include follow up on results for review activities. Reviewers communicate findings on an individual basis to the accountable procurement specialists. Comments from interviews with reviewers indicated that once agreement on the recommendation provided by CPRS is reached between the accountable procurement specialist and the reviewer, CPRS does not follow-up to ensure that corrective action is taken to address identified recommendations as they are considered by the Contracting Division to be suggestions and not mandatory for implementation.

The effectiveness and value-added of review activities may not be optimized when results from monitoring and review activities are not compiled and analyzed to identify common trends and patterns. If not corrected, identified deficiencies may erode the effectiveness and efficiency of procurement processes. The risk of missed opportunities for continuous process improvement is increased.

Recommendation

As a best practice, FAB should include a follow-up process to improve the value-added, particularly with respect to recommendations.

Action plan
In accordance with the Procurement Compliance Review Directive, where there is disagreement for high risk matters, CPRS escalates to FAB senior management.

To ensure the loop is closed regarding any mandatory change agreed to by the Contracting Authority (CA), the CA's manager will forward the proposed revisions to CPRS. The Compliance Review Report will not be signed off until the changes have been made and sent to CPRS for final review and approval.

Recommendation

FAB should support continuous process improvement by performing detailed trend analysis of results from mandatory file reviews.

Action plan
Starting in November 2013, CPRS began publishing quarterly newsletters discussing key procurement topics that had been observed when conducting Mandatory Compliance Reviews, Targeted Compliance Reviews as well as current trends. In addition, as already noted in our action plans under a previous recommendation, CPRS is working on developing Compliance Review Checklists that will better ensure consistency in reviews and allow for the performance of trend analysis of results from mandatory file reviews. Completion date: March 2015

3.3 Reporting of contracting activities

Reports prepared and provided to Agency senior management and external oversight bodies must include accurate and reliable data for informed decision-making. For CRA contracting activities, information is reported in the CRA Semi-annual Contracting Report that is presented to the Resources Committee of the Board of Management (Board).

The Agency's web-based procurement and ordering tool (i.e. Synergy) and CAS are the principal sources of data used to provide the information for the Agency's Semi-annual Contracting Report. Therefore, the transactional data in these systems for CRA procurement activities must be accurate and reliable.

Results from a review of procurement information provided in the 2012-2013 Semi-Annual Contracting Report indicated data deficiencies. Contributing factors to deficiencies include coding errors as discussed in section 2.1 of this report, such as incorrect classification of solicitation procedure (i.e. non-competitively awarded contracts reported as competitively awarded contracts and vice versa). In addition, it should be noted that 2012-2013 is the first year that transactions for ATF contracts were excluded from the report to the Board.

Recommendation

FAB should monitor key elements of procurement transactions for accuracy and reliability of information captured for reporting purposes.

Action plan
An assessment of 2013-2014 for Qualified Supplier List data found that all of the transactions were properly coded as “Traditional Non-competitive”.

AD will modify its current monitoring and validation process to improve the accuracy of CAS data elements. Completion date: March 2015

Conclusion

As explained in the File Review Methodology section, due to the coding inaccuracies of the contracts, it is improbable that the samples drawn are representative and the frequencies and other quantitative results cannot be generalized to the larger population. For the most part, for the files reviewed, the procurement transactions are in compliance with established processes and procedures.

Throughout the audit, IA has identified opportunities for improvement. As a result of the audit findings, recommendations have been made to improve internal control over compliance with all the steps required by the procurement procedures, data accuracy, the completeness of documentation in the contract files, and consistency in records management.

Management has developed action plans to address the audit findings. IA has reviewed management's action plans and concluded that they are adequate and reasonable to address the recommendations. To validate the complete implementation of these plans, IA plans to conduct a limited follow-up audit in 2015-2016.

Acknowledgement

We would like to acknowledge, recognize and thank the Finance and Administration Branch for the time dedicated and the information provided during the course of this engagement.

Appendix A - Breakdown of contract files reviewed

Appendix B - Procurement chart