Audit of Contracting for Transportation

September 2013

7050-9-4-15 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.

Acronyms and Abbreviations

Results in Brief

The Department of National Defence (DND) is one of the largest organizations in Canada that supports complex, geographically dispersed operations that require trained personnel, materiel management, and infrastructure. Contracted transportation services have played a critical role in supplementing the Department’s transportation capabilities, particularly while transporting CAF personnel and while delivering and repatriating equipment and supplies related to missions and disaster-relief operations. Sound contracting practices that are effective in supporting operational requirements and that stand the test of public scrutiny are essential for meeting objectives and maintaining the public’s confidence in the fairness and transparency of the contracting process.

The objective of the audit was to assess whether the Department’s current contracting processes for high-risk transportation contracts deliver good value¹ while complying with relevant departmental and central agency policies.

Findings and Recommendation

Framework for Developing and Awarding Transportation Contracts

The Department has significantly improved the guidance documents and contracting processes to help responsible personnel develop, award, and administer high-risk transportation contracts. Audit findings and subsequent discussions with contracting personnel did point to a need for a clear and collective understanding of how contract scope boundaries are defined and revised, while ensuring that amendments are consistently approved by the proper authorities.

Note: Please refer to Annex A—Management Action Plan for the management response to the Chief Review Services (CRS) recommendation.

Introduction

Background

The amended CRS Risk-based Audit Plan for fiscal year (FY) 2012/13 identified the need for an audit in the area of transportation contracting² to assess whether adequate processes are in place to ensure value for money and compliance with relevant policies for high-risk transportation service contracts. In 2006, CRS conducted an Audit of Contracting for Transportation in response to the recognition of weaknesses in the control framework for software and personnel services contracts. The key risks identified in the 2006 audit were tested in the current audit against contracts that qualified as exceeding the Department’s regular transportation contracting authority of $400,000.

Since 2006, the Department has been granted temporary exceptional contracting limits for transportation services, which enable it to undertake transportation activities in an efficient manner and ensure timeliness in reacting to sudden crisis events such as war or disaster-relief. A stipulation for the continuation of this exceptional contracting limit was that an audit be completed on its use.

Prior to 2006, the Department’s transportation and freight contracting limit for noncommon carriers³ was $400,000. In 2006, exceptional authority of $10 million was initially delegated to the Department for entering into transportation contracts based on a traditional competitive bidding process.⁴ The intent was to help the Department expedite contract approval processes for the use of contracted airlift until such time as the Department could acquire its own air-lift capability. Even this level of exceptional authority would have been insufficient for approving transportation contracts that often included the ability to exercise options for increasing the number of trips made by transport carriers to the zone of operations. In recognition of this fact, the exceptional authority was increased to $25 million later that same year. This exceptional authority was originally valid until June 2012 but was extended to June 2014 to facilitate any return of equipment and supplies from the zone of operations that could not be accomplished with CAF aircraft capacity.

Director General Procurement Services (DG Proc Svcs) establishes air, sea, and rail service contracts with commercial providers to support CAF operations and exercises both domestically and internationally. These transportation contracts augment existing CAF transportation capabilities. Contracts initiated by DG Proc Svcs generally involve larger sums of money, are strategic in nature, and require coordinated participation from CAF units based throughout Canada and/or abroad.

Under the exceptional contracting for transportation, Assistant Deputy Minister (Materiel) (ADM(Mat)), DG Proc Svcs, Director Major Procurement (D Maj Proc) and D Maj Proc 8 have the authority to approve a contract (including amendments) up to $25 million, $10 million, $3 million, and $1 million respectively.⁵ These authorities were in relation to those cases where services involved transportation of personnel and materiel related to the Afghanistan mission.

Objective

The objective of this audit was to assess whether the Department’s current contracting processes for high-risk transportation service contracts deliver good value while complying with relevant departmental and central agency policies.

For a detailed list of criteria associated with the audit objective and the source of the criteria, please refer to Annex B—Audit Criteria.

Scope

To provide value to the Department and meet the requirement for an audit of the use of exceptional contracting authority, it was determined that transportation service contracts greater than $400,000 (cumulatively valued at approximately $410 million⁶), and covering FY 2010/11, FY 2011/12, FY 2012/13 (to the end of September 2012) would be considered in the scope population. On a cumulative basis, the risk based audit sample consisted of 13 contracts and 35 associated contract amendments valued at $272.7 million.⁷ All 277 associated invoices were reviewed as part of these contractual agreements. These contracts and their amendments represented approximately 29 percent of the total number of contractual agreements and covered approximately 67 percent of the cumulative value of contractual agreements over $400,000⁸ in the scope FYs. Refer to Annex C—Profile of Transportation Contracts in Scope for further information on the profile of contracts in the audit scope.

Methodology

The conduct of this audit involved:

• reviewing relevant Government of Canada and DND policies, directives, and initiatives related to contracting;
• analyzing Contract Data Management System data to identify transportation contracts for inclusion in scope;
• analyzing the Defence Resource Management Information System (the departmental financial system) data to profile expenditures against transportation contracts;
• conducting interviews with technical authority, contracting authority, and contracting policy personnel;
• conducting interviews and file reviews with pre-payment verification personnel at Assistant Deputy Minister (Finance and Corporate Services);
• analyzing formal payment requisition approvals and formal confirmations of rendered services associated with these contractual agreements; and
• reviewing in detail the sampled in scope transportation contract files (original contracts, amendments, and associated invoices).

Statement of Conformance

The audit findings and conclusions contained in this report are based on sufficient and appropriate audit evidence gathered in accordance with procedures that meet the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. The audit thus conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the audit and apply only to the entity examined.

Findings and Recommendation

Since the 2006 CRS Audit of Contracting for Transportation, the Department has significantly improved the manner in which it awards, approves, and monitors transportation contracts exceeding the Department’s regular transportation contracting authority.

Guidance Documents and Documented Roles and Responsibilities

Adequate and relevant guidance documents⁹ exist at the central agency and departmental levels to provide direction on general contracting principles and procedures. These are supplemented by guidance documents at the section level that are tailored to strategic transportation contracting¹⁰ and that have been made widely available to relevant staff to help ensure good contract management practices, particularly in anticipation of high-value contracts and a general increase in contracting workload resulting from the mission in Afghanistan.

Such documents facilitate the effective and efficient handling of unique and often complicated transportation contracting requirements within the Department. As such, key departmental guidance documents have been continually undergoing changes so as to provide the most relevant and useful information to smaller contracting units or clients within the Department who wish to contract transportation services.

Awarding of Contracts

2006 Audit Finding. For air transportation contracts, there was no clear method of determining how the winning bid was selected. For example, 18 of the 87 sampled air contracts were sole-sourced without documented rationale.

Current Audit Finding. The Department has a robust process for selecting the winning bidder and for ensuring contracts are competitively sourced. For the type of contracts in scope for this audit, the source list of pre-qualified vendors (obtained via MERX¹¹) that are able to provide airlift services included 16 to 20 vendors.¹² For each contract examined, an invitation to tender was sent to each vendor on the source list,¹³ which is updated every two years with interested and qualified vendors. Contracting staff also have been reaching out to members of the transportation industry to facilitate increased competition among potential providers of contracted transportation services.

The audit identified that the technical authority and contracting authority personnel, as well as the contract coordinator, are all cognizant of the need for a high degree of coordination. All three are involved in the initial review of the statement of work, the articles of agreement, the general conditions of carriage, and any supplementary conditions for each contract.

Bid Evaluation

While relevant staff members were involved in the contract bid evaluation process, some of the contract file documentation did not consistently demonstrate this involvement. Specifically, standard operating procedures require the Bid Comparison Sheet, which is part of the contract file, to contain three separate evaluators’ signatures, including that of at least one technical authority or contract coordinator evaluator. Of the 13 bid comparison sheets examined, only four fulfilled this condition. It is important to note that only one bid was received for 12 of the 13 contracts reviewed. The remaining contract involved multiple bids and its Bid Comparison Sheet was one of the four that did contain appropriate signatures. Consistent completion of all formal bid evaluation steps by the relevant personnel demonstrates objectivity of the bid selection process and ensures that bids meet the contract requirements.

Exercising Contracting Authority

2006 Audit Finding. The DND delegated contracting authority limit was exceeded and/or the contract was not signed by an individual with proper authority in 92 out of 109 sampled air, rail, and sea transportation contracts.

Current Audit Finding. All 13 contracts examined were signed off by the appropriate DND contracting authority. The Department’s amendment approval process for transportation contracts is based on sufficient, clear, and up-to-date contract information that is made available to relevant contracting officers who appropriately exercise their amendment approval authority. Transportation contract approval limits at various levels within the Department are consistent with the Treasury Board Contracting Policy. In 45 out of 48 cases (13 contracts and 35 amendments¹⁴), contracting authority was exercised within assigned delegations. The Department also maintained a good practice of attaching summary sheets to contract amendments. These sheets provide a brief history of the contractual changes and facilitate the amendment approval decision process. Summary sheets also improve the ability to audit a contract file and can serve as a useful repository of information when setting up future transportation contracts.

Clarity on Scope of Contract Work

A clear and common understanding of the scope of contract work helps ensure that the Department obtains and pays only for the services it needs and that required approvals are granted with full understanding of expected outcomes. There is, however, no clear guidance of what constitutes a change in scope of contract work. As a result, two of the 35 contract amendments examined were approved at an insufficient authority level. The departmental standard operating procedures state that a contract amendment can be approved by an authority below the appropriate delegated authorities,¹⁵ provided that there is no increase in the contract value and no change in the scope of contract work or the negotiated cancellation fees.

Both of these amendments resulted in a net reduction of each contract value.¹⁶ In the first instance, four flights were added that subsequently terminated at a stop-over destination that was not in the original statement of work. In the second instance, an approved amendment involved the addition of negotiated charges¹⁷ that would have been considered a change in the scope of work. While these amendments resulted in a net reduction in the value of the contract, amendments that result in a change in scope must be approved by the delegated authorities as stated in the standard operating procedures.

Contract Errors

It is important to have a clear and collective understanding of the nature of an error and the manner in which contract approval authorities should be exercised to correct them. In one of the 35 amendments examined, an error should have been handled differently in regards to its approval.

A flight was erroneously added to the wrong contract via an amendment.¹⁸ Once this error was discovered, two additional contract amendments were created to correct the error.¹⁹ The first amendment to eliminate the contract amendment error was approved at the appropriate level. However, the addition of this flight to the originally intended contract unfortunately resulted in a change in scope of contract work and should have been approved as per the appropriate delegated authorities.

Pricing Structure of Contracts

2006 Audit Finding. More than half the sampled air, rail, and sea transportation contracts either did not include a contract ceiling or the ceiling had been exceeded without the benefit of a contract amendment.

Current Audit Finding. All examined original contracts and amendments contained ceilings that were reflected in contract pricing sheets.²⁰

Invoicing Against Contracts and Financial Administration Act (FAA) Section 34 Certification²¹

2006 Audit Finding. In 39 percent of the sampled contracts, invoices included charges for items not mentioned in the contract and in 24 percent of cases, the invoices did not include supporting documentation for third-party charges. FAA Section 34 certification was non-existent on paid invoices for 13 of the 109 sampled air, rail, and sea transportation contracts. The quality of the FAA Section 34 certification was questionable on a high percentage of the remaining invoices.

Current Audit Finding. In 276 out of 277 invoices, charges could be traced back to contractual details and supporting documentation. The Department’s payment approval process enabled it to provide reasonable assurance of being invoiced for services that had actually been carried out and that were in accordance with the most updated contractual agreements. Charges were generally well-supported by documentation such as emails, post-flight reports,²² and contract pricing sheets, as follows:

• All 13 original contracts and their 35 amendments had contract pricing sheets.
• Of 267 flight-related invoices,²³ 266 were backed by adequate supporting documentation to indicate that flights had been carried out.
• Authorities were in place to initiate expenditures and create spending commitments against relevant contracts.
• All contract invoices were charged to the respective financial commitments.²⁴
• Appropriate FAA Section 34 certification stamps/signatures were present on all 277 paid invoices examined.

Pre-Payment Verification Process

Prior to release of payment to the contractor, pre-payment verification personnel examine contract validity, contract pricing sheets, invoice charges, confirmation of flight completion, correctness of financial coding, correctness of vendor details, as well as other supporting documents, such as emails and third-party invoices. An examination of a sample of five high-risk, high-dollar value invoices (each from a separate contract) was conducted, and the supporting documentation confirmed that the pre-payment verification process was done in a thorough manner.

Performance Metrics

Contracted transportation services play a critical role in ensuring CAF responsiveness by supplementing the use of CAF transport capabilities, particularly during DND missions. These additional transportation resources are often required on short notice during missions given the changing circumstances and the availability of CAF aircraft. An examination of the 13 contracts revealed that the Department took an average of approximately 12 working days to contract transportation services from the moment that requirements were defined to the time the contract was actually awarded. An average of approximately 20 working days elapsed between contract award and the first flight service delivered as per the contractual agreement. These relatively short timelines demonstrate the efficiency and effectiveness of the contracting process in supporting CAF transportation requirements.

General Conclusion

Several good practices in the high-risk transportation contracting process identified during the audit were backed by a sound management framework that has significantly improved since the 2006 audit in areas of contract award, exercising of contracting authority, contract pricing structure, invoice and contract reconciliation, and FAA Section 34 certification. The Department’s pre-payment verification process of the invoices for the contracts examined is effective. The in-house administration of these contracts has been conducted in an expeditious fashion while delivering good value, something which should be more clearly demonstrated through further process enhancements. Overall, processes for developing and awarding these contracts comply with departmental and central agency policies. Slight improvements in the areas of contract work scope definition and contract amendment approvals, in addition to consistent application of formal bid evaluation procedures, will help strengthen compliance with contracting policies related to contracted transportation services.

Annex A—Management Action Plan

CRS uses recommendation significance criteria as follows:

High—Controls are not in place or are inadequate. Important issues are identified that could negatively impact the achievement of program/operational objectives.

Moderate—Controls are in place but are not being sufficiently complied with. Issues are identified that could negatively impact the efficiency and effectiveness of operations.

Low—Controls are in place but the level of compliance varies.

Performance Metrics

Management Action

Contract Amendments

• The D Maj Proc 8 Standard Operating Procedure document for Contract Amendments will be updated to include established criteria that define formally what constitutes a change in scope for a transportation contract.
• This Standard Operating Procedure will be communicated electronically and placed in the Departmental Records Documents and Information Management System for full visibility by all transportation contracting personnel.

Bid Comparison Sheets

• The D Maj Proc 8 Standard Operating Procedure for Bid Evaluation will be revised. It will continue to recommend three people participate in bid evaluations, but will add the wording “a minimum of two people” to allow for instances where, due to urgency or need, only one Contract Authority and one Technical Authority evaluate the bids. This complies with PWGSC Bid Evaluation Guidelines.
• A list of approved bid evaluation “alternates” will be developed to fill in for the Technical Authority, for instances when they may not be available to sign. Bid Comparison Sheets will be updated to include, in the signature block, the person’s designation and the date. Also, Individual Scoring Sheets will be updated to include signature blocks and the date.
• To ensure compliance with the above procedures, all files going forward for approval will include both the Bid Comparison Sheet and the Individual Scoring Sheets. These documents will be included in the approval checklist.

OPI: ADM(Mat)/DG Proc Svcs/D Maj Proc

Target Date: September 2013

Annex B—Audit Criteria

Criteria Assessment

The audit criteria were assessed using the following levels:

Governance

1. A framework is in place to ensure that the process of developing and awarding contracts for transportation services is in compliance with central agency and departmental policies.

Assessment Level 2 – Sufficient guidance documents exist to help contracting authority and technical authority personnel perform their roles during the contracting process. A clear definition of contract work scope and guidance on approval procedures for correcting contract errors will make contract amendments less vulnerable to improper approval.

Internal Controls

2. The Department’s controls are effective for managing the contract award and payment approval processes.

Assessment Level 3 – Payment approval processes for contracted transportation services exhibit good controls. The analysis of bids is done by contracting authority and technical authority personnel with a fair level of rigour. Bid evaluation practices do require some improvements in providing clear evidence on the Bid Comparison Sheet that evaluators are formally and consistently exercising their responsibilities. Such improvements will increase the transparency of the bid evaluation process.

3. The Department has an effective pre-payment verification process for high-risk transportation contracts.

Assessment Level 1 – The Department’s pre-payment verification process ensures that high-dollar, high-risk invoices related to transportation contracts undergo sufficient examination prior to the release of associated payments to vendors.

4. Contracting roles and responsibilities typically conducted by a centralized contracting authority are administered in-house in a comparable timeframe.

Assessment Level 1 – The Department’s record-keeping practices help maintain a comparable history (for each transportation contract file) of time taken, from receipt of client requirements to contract award.

Sources of Criteria

• Treasury Board Secretariat, Core Management Controls: A Guide for Internal Auditors, November 2007.
• Government Contracts Regulations.
• Treasury Board Contracting Policy.
• Delegation of Authorities for Financial Administration for DND and the CAF.

Annex C—Profile of Transportation Contracts in Scope

• There were a total of 13 original transportation contracts over $10 million and 35 associated contract amendments.
• The combined value of contracts in scope was approximately $272 million with actual associated expenditures totalling almost $258 million.
• All 13 contracts involved transportation of cargo by air and commenced either in FY 2010/11 or FY 2011/12.
• Unscheduled charges incurred due to flight cancellations, flight re-routings, de-icing, and delay damages comprised less than 2.4 percent of the total invoiced amount of contracts examined.
• All of these contracts were the result of a traditional competitive bidding process that used a source list of pre-qualified suppliers via MERX.
• All 13 contracts were won by one Canadian company that acted as the transportation broker for foreign aircraft owners/operators of specific Antonov and Ilyushin aircraft.
• Transportation under all 13 contracts related to the following major missions: Operation Athena (Sustainment), Operation Athena (Vehicle Movement), and Mission Transition.

___________________________________________________________________________________________________________________________

Footnote 1 In the context of this report, good value refers to awarding a contract to the vendor that meets the mandatory selection criteria and the most non-mandatory requirements and that provides the best price for all the requirements delivered. Additionally, it refers to whether DND makes the most efficient use of its resources to administer the contracting process. It does not consider whether the decision to contract rather than use departmental resources provides the best value for money.

Footnote 2 This audit was entitled Exceptional Contracting Authority in the CRS FY 2013/14 Risk-based Audit Plan.

Footnote 3 Non-common carriers are those that offer transportation services at contractually negotiated/non-published rates.

Footnote 4 A traditional competitive bidding process refers to bids solicited using traditional means such as fax, phone, newspapers, or source lists (not advertised on the Government Electronic Tendering System). As per the Delegation of Authorities for Financial Administration for DND and the CAF.

Footnote 5 Note that ADM(Mat), DG Proc Svcs, D Maj Proc, and D Maj Proc 8 form a hierarchical organizational structure from top to bottom in that order. The extent of contract approval authority is consistent with the organizational hierarchy.

Footnote 6 This excludes standing offers and supply arrangements. Data source: DND Contract Data Management System.

Footnote 7 The amount invoiced against all contractual agreements in scope was approximately $258 million.

Footnote 8 There were a total of 133 contractual agreements (excluding standing offers and supply arrangements) over $400,000. Data source: DND Contract Data Management System.

Footnote 9 These include the Treasury Board Contracting Policy suite, the Public Works and Government Services Canada (PWGSC) Supply Manual, the departmental Procurement Administration Manual, and the departmental Delegation of Authority for Financial Administration.

Footnote 10 These include standard operating procedures developed in FY 2009/2010 and a matrix outlining the distinct responsibilities of relevant personnel involved in the contract award and management process.

Footnote 11 MERX was the former internet-based electronic tendering system used by the federal government to advertise contracting opportunities to potential bidders (Delegation of Authorities for Financial Administration for DND and the CF – May 1, 2012). On June 1, 2013, Buyandsell.gc.ca/tenders became the official source for Government of Canada tender notices.

Footnote 12 The contracts examined required aircraft capable of moving a maximum amount of cargo in a minimum number of trips given the limited availability of landing slots shared with other allied nations during the Afghanistan mission. In 12 of the 13 contracts in the audit sample, this limitation may have resulted in the reduction of the number of bidders to just one vendor. A second company capable of meeting the contract requirements was added to the source list in April 2011 and did submit its bid for one of the 13 sampled contracts. Invitations to tender were sent to the entire source list of 16 to 20 vendors for all 13 contracts sampled in the audit.

Footnote 13 This requirement is referenced in D Maj Proc 8’s Standard Operating Procedures.

Footnote 14 These amendments were either put in place to help exercise original contract options (for carrying out contracted flights at a future date) or to formally lay out other charges (such as flight cancellations, flight re-routings, damages due to delay, de-icing charges, etc.) as per the most recently agreed upon contractual terms.

Footnote 15 These limits are referenced in the Delegation of Authorities for Financial Administration for DND and the CAF.

Footnote 16 These amendments reduced their contract values by approximately $1.5 million and $111,000 respectively.

Footnote 17 Charges negotiated between the Department and the contractor represented items that were not referenced (in either monetary or non-monetary terms) in the most recent scope of contract work. In this instance, the negotiated charges for damages due to delay would have been considered a change in scope and they amounted to $95,000 on a contract worth almost $19.4 million.

Footnote 18 This amendment was approved at the appropriate level of contracting authority.

Footnote 19 The first contract amendment removed this flight from the wrong contract and the second contract amendment added this flight to the intended contract.

Footnote 20 Contract pricing sheets detail the nature of charges for which the contractor could potentially bill.

Footnote 21 A Section 34 certification stamp is used by the lead technical authority to certify that the transportation services delivered, as well as the associated charges, are in accordance with contractual terms and conditions.

Footnote 22 A post-flight report for cargo flights contains information such as the date of flight, flight number, route, scheduled and actual departure/arrival date and time, and actual payload.

Footnote 23 Of the 277 invoices associated with the 13 contract files examined, 267 were flight-related invoices.

Footnote 24 As per the DND Director Financial Policies and Procedures, a commitment is an obligation to make a payment as a result of an agreement made in good faith with other parties. As per the Department’s Financial Administration Manual, Chapter 1016-2, commitment authority refers to the authority delegated by the Deputy Minister to incumbents of positions to confirm the availability of funds before entering into an agreement.