Audit of Contracted Facilities Maintenance Services

Back to Table of Contents

1.2 Rationale

Given that DND/CAF’s infrastructure is aging and the recent transformation that resulted in ADM(IE) becoming responsible for all departmental infrastructure and related maintenance, ADM(RS) conducted an audit of Contracted Facilities Maintenance Services. The audit was included in the ADM(RS) Risk-based Internal Audit Plan for fiscal years 2016/17 to 2018/19. Findings and recommendations in this report can inform ADM(IE) decision making in support of its centralized portfolio management approach to infrastructure.

1.3 Objective

The audit objective was to determine whether contract arrangements for FM services are managed to achieve VFM, ensure continued infrastructure availability and sustainability, as well as comply with environmental, health and safety regulations.

1.4 Scope

The scope of this audit included locations that had contracted FM services during fiscal years 2016/2017 and 2017/2018. This audit was conducted between the months of October 2017 and March 2018.

The audit team did not determine if VFM was achieved, but rather looked at if both ADM(IE) and ADM(Mat) have the required information to determine if VFM was achieved.

1.5 Methodology

A non-statistical sample of five contracts for FM services was chosen for examination based on dollar value, geographic diversity and suggestions from ADM(IE). The sample was selected from a population of 36 contracts provided by ADM(IE) as the known inventory of both FM only and FM full service contracts. The population of 36 contracts included six full service contracts, 27 FM only contracts and three contracts that were undefined as per the information provided by ADM(IE).

The FM value of the five contracts examined in our review represented about 50 percent of the total FM value of both FM only and full service contracts ($131 million out of $258 million).

The audit used the following methodology:

• File and document review, including contracts, performance reports, inspections reports, and committee minutes;
• Interviews with key internal DND/CAF stakeholders (e.g., Commander of RP Ops Gp within ADM(IE), RP Ops Gp personnel, local level contract management cells and ADM(Mat) Director of Procurement); and
• Site visits to the five sites with contracted FM services (five sampled contracts) :
  • The North Warning System (managed out of the National Capital Region);
  • Canadian Forces Base Goose Bay;
  • Land Force Central Area Training Centre Meaford;
  • Toronto Armouries; and
  • Defence Research and Development Canada – Toronto.

1.6 Audit Criteria

The audit criteria for the audit were as follows:

• Decisions related to contracted FM services are fully supported and properly approved.
• Agreements for contracted FM services ensure infrastructure is available to support operational requirements.
• Requirements for contracted FM services ensure compliance with environmental, and health and safety regulations.

The audit criteria can be found at Annex B.

1.7 Statement of Conformance

The audit findings and conclusions contained in this report are based on sufficient and appropriate audit evidence gathered in accordance with procedures that meet the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing. The audit thus conforms to the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the audit and apply only to the entity examined.

Back to Table of Contents

2.0 Findings and Recommendations

2.1 Information for Decision Making

To make informed decisions, relevant, timely, accurate and accessible data is required. For the infrastructure services that are contracted, required data may include condition of assets, SOW and Additional Work Requests^{Footnote 2}  (AWR) contract costs and contractor performance results.

We expected ADM(IE) to have defined information needs and performance indicators, and, at a minimum, to gather and report the following information:

• Consolidated details of all contract arrangements for FM;
• Analysis of the cost of FM options (e.g., contracted resources versus internal resources);
• Assessment of contractor performance; and
• Contract costs (including SOW and AWR).

FM contracts are currently managed differently across locations resulting in varied information being gathered. This limits the ability of the Department to collect consistent FM information and identify lessons learned to inform future FM decisions.

2.1.1 Consolidated information

ADM(IE) does not currently gather consolidated information on contracted FM arrangements. A spreadsheet of all full service and FM only contracts and their estimated values was manually created by RP Ops Gp for the purposes of this audit. The data provided was incomplete and inconsistent (e.g., total contract dollar value versus annual contract dollar value). As a result, the audit team was not able to verify the accuracy of the information provided. Further, for the sampled contracts, there was no evidence of any analysis on how much was spent on AWRs that were over the established SOW ceilings. Only one site was able to provide a partial analysis by requesting an ad hoc report from the contractors’ systems. Creating this report was labour intensive given that the data was not easily accessible. Not having access to accurate information does not allow ADM(IE) to have a complete view of FM services.

Between DND and the various contractors, there is a variety of systems used to gather data such as DND specific systems, contractor specific systems and others. For security reasons, not all contractors have access to DND systems. Using such a variety of systems increases the risk of inconsistency of data and its comparability, as well as the risk of human error in the amalgamation of information across various locations.

Interviews indicated that full service and FM only contracts have little to no FM information, performance indicators or data analysis requested by the chain of command. Any FM data collected remains at the local level. Without clear data needs identified, it is difficult to collect relevant information for decision making in relation to the ADM(IE) infrastructure portfolio. The new module in DRMIS will consolidate all infrastructure information such as asset condition, assessment and replacement costs. At the time of this audit, two of the contracts reviewed had already entered basic information in DRMIS. Interviewees indicated that enhanced DRMIS related training was required to better utilize the system for FM services management. Implementing this module will allow the Department to address some of the aforementioned risks.

2.1.2 Value-for-Money

The Department has varied interpretations and definitions for VFM, with different short-term and long-term focuses. For example:

• From a performance management perspective, VFM may be the contractor’s overall performance of FM services.
• From an infrastructure perspective, VFM may be the long-term sustainability and recapitalization costs of infrastructure.
• From a contracting perspective, VFM may be the cost of the contract (e.g., lowest bid).

2.1.3 Cost-Benefit Analysis

While interviewees indicated that ADM(IE) does not currently have the capacity to internally maintain all departmental infrastructure, there was no evidence of the Department gathering data to analyze and compare costs of in-house FM versus contracted FM services. This increases the risk of ADM(IE) not having the necessary information to make any future “make or buy” decisions on FM services.

2.1.4 Contractor Performance

Past contractor performance is important when determining whether to exercise current contract options or negotiate new contracts. There was limited evidence of analysis at the local or portfolio level (such as analysis of past quality of work or QA reports). At one site visited, a sustainment business case recommended not exercising the contract option with the current contractor. While the business case outlined how the contractor was not meeting desired performance requirements, there was no evidence of further advancement of the sustainment business case. Not formally assessing contractor performance limits the ability of the Department to identify lessons learned and effectively negotiate contract terms and conditions that better meet departmental expectations.

Attractive contract terms and conditions aligned with location specifics could provide the Department with a greater number of bidders. For example, short term contracts may not be beneficial for contractors to competitively bid on FM only or full service contracts. A small pool of bidders increases the risk of having a less qualified contractor maintain DND/CAF infrastructure.

Revenue-at-risk clauses allow the Department to withhold a portion of the fixed contract cost for work that is below standard or not fully completed. Four of the five contracts sampled contained such a clause.

PIFs are a key mechanism which allow the Department to periodically assess contractor performance. PIFs are intended to promote cost-saving innovations and incentivize excellence by the contractor. While four of the five contracts sampled have PIFs based on key performance indicators (KPI), they were awarded differently in each of the contracts. In one full service contract, the amount of PIF awarded was based on the contractor’s performance of all the SOW, using a weighted system. In another full service contract, the PIF payment was based on work that was over and above the SOW (e.g., AWRs), but did not assess if the SOW was completed. There is a risk that the full fixed price of the contract and the PIF could be awarded without meeting all the SOW requirements. With PIFs being awarded differently, contract performance may not always meet the infrastructure maintenance and sustainability goals of ADM(IE).

2.1.5 Contract Costs

Clear wording of contract requirements, such as the SOW, helps ensure all parties involved understand the work to be completed. The current wording of SOW technical requirements in the contracts sampled had varying amounts of detail. The risk of being too prescriptive is that if a component is missed in the SOW, the contractor could submit an AWR, resulting in an increased cost to the Department.

The long-term sustainability and recapitalization costs of infrastructure are an important consideration in the decision to contract FM services. This is not always reflected in the SOW. For some contracts, the focus was on keeping the fixed contract costs low and did not always consider the long-term infrastructure sustainability. For example, in one sampled contract, the SOW states that the infrastructure is to be available to the extent required to support normal operations. Interviewees expressed concern that a contractor could be doing the minimum required with little to no preventative maintenance to ensure its long-term sustainability. A reactive approach, such as this, could increase the risk that infrastructure would require major repairs or replacements sooner than expected.

While having a low bid price is another criterion for VFM, it can lead to AWRs being required to meet the actual FM infrastructure needs. AWRs are included in the contract as a legitimate contractual tool, enabling the Department to ensure the long-term sustainability of its infrastructure requirements. Interviewees at one of the five sites visited suggested that a contractor bid significantly lower than the other bidders to win the contract. There have since been issues with the contractor in completing the SOW and a number of AWRs have been issued.

AWRs occurred in every contract sampled leading to additional unforeseen costs. If not closely managed and without the consistent use of well-defined parameters, certain contracting risks may exist for AWRs, such as:

• Perception of sole sourcing as other contractors are unable to bid on additional tasks included in the SOW;
• Taking time away from SOW work from both a contract management perspective and contractor perspective as AWRs generate extra revenue for the contractor, above the fixed price contract;
• Overall more costly approach to maintaining infrastructure; and
• Having difficulty budgeting the true cost of FM.

Back to Table of Contents

Back to Table of Contents

2.2 Authorities, Roles and Accountabilities

Back to Table of Contents

2.3 Environmental, Health and Safety

We expected that the Department would comply with the Environmental, Health and Safety requirements listed in each contract.

2.3.1 Environmental Management

Based on SOW requirements, the contractor reports environmental issues (e.g., spills of chemicals), as applicable. Reporting by the contractor’s environmental officer is coordinated through the Department’s environmental officer. Contractors are responsible for remediating all spills of chemicals or HAZMAT-related issues regardless of who caused the spill. Interviews and environmental reports indicated that environmental issues and corresponding remedies have been reported to DND. Follow ups by an independent third party are completed the year following an audit, if necessary, to ensure any corrective measures recommended have been implemented and are in compliance with environmental standards.

2.3.2 Health and Safety

As per contracts, the Department and the contractor are respectively responsible for the health and safety of their own personnel. The audit found no issues in this area.

Back to Table of Contents

3.0 General Conclusion

Contracted FM services are managed to ensure infrastructure needs are met in the short term as well as comply with environmental, health and safety regulations. To help ensure consistency in FM management, VFM and the long-term sustainability of infrastructure, ADM(IE) will need to identify information requirements and establish and communicate clear FM related ARA.

ADM(IE) has recently become the custodian of all DND infrastructure. The different authorities and roles across FM related contracts are not aligned with the new accountabilities of ADM(IE). This has resulted in varied QA practices and inconsistent FM management which has limited ADM(IE)’s ability to optimize its oversight of the infrastructure portfolio.

ADM(IE) requires additional FM information from each location to help leverage lessons learned and ensure increased comparability and accuracy of information across the infrastructure portfolio. Standardized ARA for FM stakeholders and a risk-based QA approach will provide more consistent oversight of FM.

ADM(IE) may wish to consider using the lessons learned from current FM related contracts to develop consistent SOWs, PIFs and other clauses in FM related contracts that meet the Department’s needs and potentially reduce expenses.

Back to Table of Contents

Annex A - Management Action Plan

ADM(RS) uses recommendation significance criteria as follows:

• Very High—Controls are not in place. Important issues have been identified and will have a significant negative impact on operations.
• High—Controls are inadequate. Important issues are identified that could negatively impact the achievement of program/operational objectives.
• Moderate—Controls are in place but are not being sufficiently complied with. Issues are identified that could negatively impact the efficiency and effectiveness of operations.
• Low—Controls are in place but the level of compliance varies.
• Very Low—Controls are in place with no level of variance.

Information Needs

Management Action

Action 1.1 – Agreed. In order to determine and communicate which FM data and performance indicators are needed to make informed FM decisions, ADM(IE) will:

• Determine and document the appropriate FM KPI levels for its infrastructure portfolio;
• Consult internal and external stakeholders; and
• Communicate strategy to all relevant stakeholders.

OPI: ADM(IE) – CF RP Ops Gp
Target Date: December 2019

Information Collection Mechanism

Management Action

Action 2.1 – Agreed. ADM(IE) will implement a mechanism to gather data, including cost information, for FM services. Specifically, ADM(IE) will develop tools and plans in order to:

• Consolidate details of all outsourcing arrangements for FM and FM related outsourcing costs. This consolidation data should all be housed within DRIMIS/IERIS; and
• Assess contractor performance and contract costs at least annually during the contract.

OPI: ADM(IE) – CF RP Ops Gp
Target Date: December 2019

Cost-Benefit Analysis

Management Action

Action 3.1 – Agreed. ADM(IE) will conduct a comprehensive cost-benefit analysis of in-house versus outsourced FM services to inform the cost efficiency of future FM services, including the renewal of FM contracts and related decisions by undertaking the following actions:

• Create an evaluation framework for cost-benefit analysis;
• Consult internal and external stakeholders;
• Analyse the cost of FM options (e.g., outsourcing versus internal resources);
• Document and communicate the evaluation framework to all internal stakeholders; and
• Apply the evaluation framework to all future FM services decisions.

OPI: ADM(IE) – CF RP Ops Gp
Target Date: December 2020

Information Collection Mechanism

Management Action

Action 4.1 – Agreed. ADM(IE) will validate the training requirements and coordinate the delivery of updated training for the Real Estate module in DRMIS to assist in the management of FM services.

OPI: ADM(IE) – CF RP Ops Gp and DGIEES/DAES/IEBDS
Target Date: March 2020

Authorities, Roles and Accountabilities

Management Action

Action 5.1 – Agreed. ADM(IE), in collaboration with ADM(Mat), will establish clear ARA as articulated in the recommendation, especially regarding contract negotiations, PIF assessment boards and quality assurance.

Additionally, ADM(IE) will review existing full service contract (that include FM) templates and modify to ensure that any future contracts will have clear ARA as articulated in the recommendation.

ADM(Mat) will be included in any policy development of FM contracts for which ADM(Mat) is implicated, and will endorse any portion of such policy that relates to ADM(Mat) prior to being implemented and/or communicated within the Department.

OPI: ADM(IE) – CF RP Ops Gp
OCI: ADM(Mat)
Target Date: April 2019

QA Guidance

Management Action

Action 6.1 – Agreed. ADM(IE) will develop and implement centralized QA FM guidance. Specifically, working with the contracting authority, ADM(IE) will:

• Establish and communicate guidance for all infrastructure asset classes that, at a minimum:
  • Identify planning methodology based on condition of asset;
  • Outline QA performance and reporting requirements;
  • Identify training requirements and expectations;
• Develop risk-based QA plans considering infrastructure sustainability, long-term operational needs, risk tolerance, and dollar value;
• Consult with stakeholders;
• Communicate policy change with internal and external stakeholders; and
• Ensure the new authorities and responsibilities related to the QA of the FM portion of FM contracts are aligned to new infrastructure accountabilities.

OPI: ADM(IE) – CF RP Ops Gp
Target Date: December 2020

Back to Table of Contents

Annex B—Audit Criteria

Criteria Assessment

The audit criteria were assessed using the following levels:

Assessment Level and Description

• Level 1—Satisfactory
• Level 2—Needs Minor Improvement
• Level 3—Needs Moderate Improvement
• Level 4—Needs Significant Improvement
• Level 5—Unsatisfactory

Governance

1. Decisions related to contracted FM services are fully supported and properly approved.

Assessment Level 4 – The ADM(IE) transformation has created two reporting structures which have created inconsistent oversight of the contractors at the Base/Wing level. There is also a lack of information to support contracting decisions.

Risk Management and Control

2. Agreements for contracted FM services ensure infrastructure is available to support operational requirements.

Assessment Level 3 – While there are terms in the contracting agreements that outline some preventative maintenance requirements, the contracts could be modified to emphasize the long-term sustainability of the Department’s infrastructure.

3. Requirements for contracted FM services ensure compliance with environmental, health and safety regulations.

Assessment Level 1 – Compliance with environmental regulations was the responsibility of the contractor with departmental oversight. Compliance with health and safety was managed respectively by the contractor and the Department for their employees. Both were compliant and no issues were identified.

Sources of Criteria

Governance:

• Reference to: Committee of Sponsoring Organizations of the Treadway Commission (COSO), Internal Control – Integrated Framework, March 2013
• Reference to: ADM(IE) transformation documentation

Risk Management:

• Reference to: COSO, Internal Control – Integrated Framework, March 2013

Controls:

• Reference to: COSO, Internal Control – Integrated Framework, March 2013

Back to Table of Contents