Final Report - Follow-up Audit of NIHB - Medical Transportation

December 2012

Management Response and Action Plan

Table of Contents

• Executive summary
  • 1. Background
  • 2. Audit objectives
  • 3. Scope
  • 4. Approach
  • 5. Statement of assurance
• B - Findings, recommendations and management responses
  • 1. Follow-up on the 2010 audit recommendations
    • 1.1. Progress made on 2010 audit issues
  • 2. Governance
    • 2.1. Strategic direction
    • 2.2. Roles and responsibilities
    • 2.3. Performance management
  • 3. Risk management
    • 3.1. Risk management
  • 4. Internal control
    • 4.1. Information systems
    • 4.2. Recipient monitoring
    • 4.3. Compliance with FNIHB guidelines
    • 4.4. Client and service eligibility
    • 4.5. Payment support
• C - Conclusion
• Appendix A - Lines of enquiry and audit criteria
• Appendix B - Scorecard
• Appendix C - Information technology (IT) application control testing
• Appendix D - Overview of Medical Transportation Data Store sources
• Appendix E - Medical transportation expenditures and systems by region
• Appendix F - Status of planned actions against 2010 recommendations
• Appendix G - Assessment Rating Guide

Executive summary

The First Nations and Inuit Health Branch (FNIHB) manages the medical transportation benefit. This benefit is available to approximately 900,000 eligible First Nations and Inuit clients to enable them access to health services that cannot be obtained in their community of residence.  To best service the First Nation and Inuit clients, administration of the benefit happens via three different delivery models. Medical transportation benefits are administered by either First Nations or Inuit organizations under contribution agreements (482 agreements across Canada); by territorial governments under a contribution agreement; or by Health Canada regional offices through operating budgets (referred to as "operationally managed benefits"). In 2011-12, expenditures totalled $333.3 million.

The focus of the current audit was to assess the effectiveness of the actions taken by management to address the recommendations from the 2010 Audit of the Non-Insured Health Benefits - Medical Transportation, and secondly; to re-examine the medical transportation management control framework at headquarters and the regions.

The 2010 audit report had seven recommendations designed to strengthen the management control framework of the medical transportation benefit. The follow up portion of the audit noted good progress by management to implement the commitments made. More specifically, management actions related to six of the recommendations have been fully implemented. The follow up noted limited progress on the recommendation related to obtaining complete, accurate and timely nation-wide data. The examination of the management control framework reveals that issues also exist in relation to system vulnerabilities, recipient monitoring, consistency in the funding approach and clarity of the terms and conditions of services provided by suppliers and paid for by Health Canada.

This report includes eight recommendations to further aid in strengthening the management control framework for medical transportation benefit. Management agrees with the recommendations and has prepared an appropriate action plan.

1. Background

Medical Transportation Benefits

• Ground travel (vehicle, taxi, bus, train, snowmobile and ambulance)
• Air travel
• Water travel
• Living expenses (meals and accommodations)
• Transportation costs for health professionals
• Emergency transportation
• Escort transportation and living expenses
• Transportation to addictions treatment and traditional healers

The Non-Insured Health Benefits (NIHB) Program, which is part of the First Nations and Inuit Health Branch (FNIHB), funds medical transportation (MT) benefits to assist eligible First Nations and Inuit to access medically required health services that cannot be obtained in their community of residence.  The Program acts as a payer of last resort in providing MT coverage by funding benefits that are not covered by other public or private insurance plans. MT benefits are provided to all registered First Nations and Inuit (approximately 900,000 in 2012), regardless of income or province/territory of residence. As a national program, NIHB benefits are portable across the country.  NIHB pays for travel to provincially/territorially insured services (for example doctor visits, diagnostic tests, hospital treatment) as well as travel to access benefits covered under the NIHB Program (for example, dental, vision care and mental health).  NIHB also covers travel for health professionals to provide services in communities when deemed cost effective.

MT benefits are administered by either First Nations and Inuit organizations under contribution agreements; by territorial governments under memoranda of agreement; or by Health Canada regional offices through operating budgets (referred to as "operationally managed benefits"). In 2011-12, 482 contribution agreements were in force for MT.

MT expenditures totalled $333.3 million in 2011-12, an increase of 6.9 percent over the previous year and represented 31 percent of NIHB's expenditures. (See Appendix E) Contribution agreements represented $153.7 million (46.1 percent) of the total benefit expenditures; and operationally managed benefits amounted to $179.6 million (53.9 percent). Drivers for expenditure increases in the MT costs include: the rising cost of fuel, the impact of rationalization of health services by provincial governments on distances travelled for services, population growth, and the recent additions to eligible populations due to a court decision and a newly recognized Band.

MT activities are primarily managed in regional offices. Until recently, all regional staff in Health Canada reported to the Regions and Programs Branch.  As a result of a reorganization that took place in December 2011, NIHB regional staff, including those administering MT, now report directly to the Assistant Deputy Minister of the FNIHB.  There have been no other structural changes or major program changes to MT delivery since the 2010 audit.

2. Audit objectives

The objectives of the audit were to determine whether:

• the implementation of the management action plan has been effective in addressing the recommendations made in the Audit of the Non-Insured Health Benefits - Medical Transportation (October 2010); and
• the management control framework for medical transportation remains effective in managing key risks.

Lines of enquiry and audit criteria are provided in Appendix A.

3. Scope

The audit examined the current management control framework at headquarters and in the regions and examined transactions recorded in fiscal year 2011-12. The examination of regional operations primarily involved the Manitoba and Ontario regions, as they account for the highest proportion of total MT expenditures (30% and 17% respectively) and they were visited and assessed in the original audit. The Alberta and Saskatchewan regions were covered via a questionnaire and follow-up interviews.

As a follow-up to the previous audit and in response to the identified weaknesses in information systems, specific controls in the Medical Transportation Records System (MTRS) and the Ontario Medical Transportation System (OMTS) were examined. Additionally, change management processes used for the OMTS were included in the scope of the audit. The work performed does not constitute a full information technology audit of these systems.  The Accurate Financial XChange Ltd. (AFXL) System used in the Alberta region was excluded from the scope of this audit, since the audit work focused on the Ontario and Manitoba Regions.

4. Approach

The audit methodology included interviews, document review and system testing both at headquarters and in the Manitoba and Ontario regions. Regional visits, which were conducted at Health Canada's Winnipeg Office and the Sioux Lookout Zone Office (part of the Ontario region), included the testing of a sample of transactions recorded for contribution agreements and operationally managed benefits. Questionnaires were conducted in the Saskatchewan and Alberta regions to gather a broader appreciation of issues dealing with governance (for example, performance measurement and the impact of the recent reorganization), risk management (for example, identification of risks at the regional level) and controls (for example, information gathering and multi-system environment).

5. Statement of assurance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Further, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing.

B - Findings, recommendations and management responses

1. Follow-up on the 2010 audit recommendations

1.1. Progress made on 2010 audit issues

Audit criterion: Management's actions have been effective in addressing the issues identified in the audit tabled in October 2010.

The Non-Insured Health Benefits (NIHB) Program has strengthened its management control framework for the delivery of medical transportation (MT) by addressing effectively the majority of recommendations made since the 2010 Audit of the Non-Insured Health Benefits Medical Transportation. Appendix F shows the efforts by management for each action item identified in its Management Response and Action Plan after the 2010 audit. In particular, this audit noted that a risk management process related to medical transportation is in place, with a mitigation strategy. Secondly, the Ontario Medical Transportation System has been modified with the addition of batch file controls. Thirdly, NIHB guidelines have been clarified in terms of the frequency of reporting required from recipients based on risk profiles. This clarification has led to some improvements in the frequency of reporting by high risk recipients. It was also observed that the claims verification process is clear, well documented and is operating effectively.

While most issues identified in the 2010 audit have been addressed, there are outstanding issues. Specifically, the collection of information for decision making remains an significant challenge given the complexity of the numerous systems used in management of medical transportation benefits. Secondly, while the Medical Transportation Records System has been modified to generate access logs that track the author and timing of changes, the nature of the changes is not identified. Finally, some recipients report expenditures by category. However, some recipients still only report the total expenditures

2010 Recommendation 1: Risk Management and Mitigation Strategy

The 2010 audit recommended that the NIHB Directorate document its program risks, including external risks, and develop mitigation strategies and monitoring activities. Section B 3.1 of this report describes the risk management process, which is satisfactory. NIHB plans to review the MT risk assessment on an annual basis.  The recommendation has been fully addressed.

2010 Recommendation 2: Collection of information for decision making

The 2010 audit reported that the information systems supporting the delivery of MT benefits did not allow for the collection of complete, reliable and timely information for decision making. In the 2010 MRAP, NIHB planned to integrate all MT systems, without a specific timeline. The NIHB Directorate undertook to produce a Multi-Year Data Collection Plan and produced data collection targets but these have not been completed. In 2005, as an attempt to integrate all data on MT, NIHB developed the Medical Transportation Data Store (MTDS) to gather data from its different sources however the complexity of MT data sources feeding into the MTDS remains (see Appendix D).

There are several factors inhibiting the collection of information. A multitude of definitions, formats and systems used to capture data on individual trips coupled with the incompatibility of systems with the MTDS, and issues with the completeness of the data collected, in particular from contribution agreement recipients. A solution should involve both the Corporate Services Branch that provides information technology services and the Chief Financial Officer Branch, for the financial requirements. A new recommendation is issued in this report to that effect.

2010 Recommendation 3: Controls over access logs in the MTRS and batch files in the OMTS

In the Ontario region, the deficiencies with the Ontario Medical Transportation System (OMTS) have been addressed as proposed in the 2010 MRAP and have been fully resolved. The MTRS has been modified to generate access logs that track the author and timing of changes, [sensitive security-related information removed in accordance with the exemption provisions of the Access to Information Act]. The actions planned against recommendation 3 have been implemented. However the MTRS access log function requires more improvement (See section B4.1).

2010 Recommendation 4: Reporting of expenditures by contribution agreement recipients

In 2010, the reporting conditions outlined by the NIHB Directorate required only the aggregate reporting of annual expenditures. The NIHB Program reviewed the reporting guidelines in the NIHB Medical Transportation Contribution Agreement Manual, as per its 2010 MRAP. Currently, not all recipients are providing sufficient financial information to allow program managers to determine if funds are spent appropriately and the primary causes for deficits. Management has fully implemented its planned actions on this recommendation. However, the reporting of expenditures without details by contribution recipients is still common (see Section B4.2).

2010 Recommendation 5: Guidelines for frequency of reporting

Prior to the 2010 audit, recipient reporting requirements had been set at three times a year and this continues with the current contribution recipient risk assessment tool. However, interviews and testing showed that program managers increased the frequency of reporting in high risk files as per the NIHB Medical Transportation Contribution Agreement Manual and were submitting 'Deviation Requests' for approval by the NIHB Directorate. Therefore, the recommendation has been fully addressed.

2010 Recommendation 6: Monitoring of activities

The 2010 audit observed that monitoring activities conducted in Ontario primarily involved ensuring that recipient reports were received on time and that cost totals matched other sources of information. The 2010 audit recommended stronger monitoring activities in the areas of financial reporting, documentation of site visits, and early intervention in case of significant issues, such as deficits in Ontario communities. During this Audit, interviews and sample testing in Ontario revealed a high level of consciousness for recipient monitoring and scrutiny over higher risk recipients. The actions taken by management have resulted in significant progress regarding monitoring of activities since the 2010 audit. Therefore, the recommendation has been fully addressed.

2010 Recommendation 7: Claims Verification Process

During the 2010 audit, file review showed that evidence of client referrals (used to obtain medical intervention outside of the community) and confirmation of attendance at medical appointments were not always on file in Ontario. In the same region, it was also found that roles and responsibilities of the employees involved in the claim verification process and Section 34 of the Financial Administration Act (FAA) needed to be better defined. This audit concludes that the process is currently well documented. Furthermore, in the samples tested, there were no problems detected. Therefore, the recommendation has been fully addressed.

2. Governance

2.1. Strategic direction

Audit criterion: The organization's strategic direction and objectives are implemented through operational objectives and priorities.

The Medical Transportation 3-Year Business Plan (2006-2009) included MT strategic priorities, measurable success factors, targets, operational plans, roles in completing the plan and accountability including reporting to senior management.

More recently, the NIHB Program has used a project management approach where MT projects for the upcoming fiscal year are identified and progress is monitored through a program work plan. However, there is no evidence of progress monitoring toward strategic objectives (outlined in the three-year business plan) since 2006.

A new MT strategic planning exercise is currently underway and is in the early stages of development. It revisits long standing issues such as the multiplicity of information management systems and uniformity of MT delivery.

Timely input from the regions in setting strategic direction is advisable as they have the most direct contact with clients and with stakeholders (First Nations and Inuit health authorities, and provinces/territories). Headquarters advises that regional participation is planned as part of the strategic planning exercise.

In conclusion, NIHB has started the process to identify strategic direction and objectives but it is too early to conclude on the extent to which these have been implemented through operational objectives and priorities.

2.2. Roles and responsibilities

Audit criterion: Roles and responsibilities are documented, clear and understood by NIHB staff.

This audit reviewed FNIHB's organization charts, the role of NIHB's headquarters, the mandate of the NIHB Managers' Forum, and questioned MT staff in four regions and at headquarters to determine if they were given enough guidance to understand their responsibilities.

The FNIHB organization chart was updated in May 2012 and reflects changes in the departmental structure. Until December 2011, regional staff, including MT delivery staff, reported to the Assistant Deputy Minister of the Regions and Programs Branch while NIHB headquarter staff were part of FNIHB. Now, both regional staff and headquarters staff delivering First Nations and Inuit health programs belong to FNIHB. Job descriptions have not changed as a result of the restructuring as the responsibilities have remained unchanged. The role of NIHB's headquarters for policy development and overall coordination remains unchanged.

The NIHB Managers' Forum serves as a link between the regional offices and central NIHB staff headquarters. It is an oversight body chaired by the Director, Benefit Management Division (part of NIHB headquarters), which meets monthly through conference calls and twice a year in person. Membership includes the NIHB regional and headquarters managers/directors or their representatives. The NIHB Managers' Forum is supported by the MT Working Group whose mandate is to identify policy gaps to assess medical transportation services, to discuss emerging issues and to provide recommendations for more cohesive policy and practical solutions to MT benefit operations.

The regional managers that were interviewed in June 2012 felt that roles and responsibilities were clear and well defined and believed that the new reporting lines would make communication with NIHB headquarters easier.

In conclusion, roles and responsibilities are documented, clear and understood by NIHB staff.

2.3. Performance management

Audit criterion: Senior management identifies performance measures, monitors performance and adjusts as required.

Performance measures

Planned results (outputs and outcomes) and performance measures for the NIHB Program as a whole were developed in the context of a Treasury Board Submission that was submitted in December 2010 in order to consolidate NIHB authorities.

Regions involved in the audit identified performance measures related to service standards such as average time to reach a benefit analyst at regional call centres, response time to complaints, and whether supplier payments were issued within 30 days of receipt of invoices. Delivering MT benefits within budgets was identified by the regions and by headquarters for monitoring and adjustments where necessary.

National level measures for the MT benefit have been identified and these are aligned with FNIHB's branch-wide performance measures as required for reporting under the Program Activity Architecture, the Departmental Integrated Operating Plan (DIOP), and the Departmental Performance Report. However, the national MT measures are based on inputs (for example, number of eligible clients), outputs (for example, number of policies reviewed/revised/issued) and usage data (number of clients that received MT services) rather than performance measures. Headquarters identified the budget variance reports ("Monthly Variance Report") as the current MT performance measurement method. However, in order to promote and measure improvements in the delivery of the MT benefit, additional measures related to program efficiency (for example, percentage of coordinated travel for routine health care) should be identified.

Performance monitoring

In order to undertake performance monitoring, NIHB must develop systems to access the data that is currently collected across the country. As noted in Section B1.1 (2010 recommendation 2), little progress has been made to allow the extraction of data from the different data collection systems shown in Appendix D. The web of data collection systems in diverse formats based on different rules impairs the Program's ability to collect timely, reliable, consistent and valid MT data which would allow the Program to measure performance data consistently across the country.

NIHB reports actual 2009-10 and 2010-11 results for some of the identified MT measures, but had to rely on data gathered from SAP, the departmental financial system. Only a few of the measures, such as usage data, rely on the MTDS for data. However, the MTDS does not have timely or complete national data. MTDS data on activities for 2010-11 is only available for 70% of the operationally managed benefit expenditures and only 10% of the benefits managed through contribution agreements. In fact, the percentage of travel data available in the MTDS is a performance measure for MT that is identified in Health Canada's DIOP.

With regards to operationally managed benefits, data is gathered at the regional level through the Medical Transportation Records System (MTRS) for all regions, with the exception of Alberta and Ontario, each of which maintain their own systems.

As for benefits managed through contribution agreements, data is collected by recipients (the communities) and sent to the regional offices. Some of this data is reported through spreadsheets. However, these spreadsheets have evolved with inconsistent data format, which has rendered the uploading to MTDS time consuming. In addition, other data systems have been developed (Manitoba bands, Nunavut, Northwest Territories) to capture activities covered by contribution agreements. The capacity of some communities to produce reports periodically while using a consistent format is also a challenge.

A significant impediment to change is the different accountabilities, roles and responsibilities of the players involved in the solution to these challenges. Within FNIHB, the role of the NIHB Directorate is to gather regional requirements and develop policies to oversee the development of NIHB's MT data systems so they can be used to collect quality and timely data to aid or contribute in decision making. Also within FNIHB, regional operations use NIHB's MT data systems to deliver and manage the MT benefit at a client-level. Under the Corporate Services Branch (CSB), the Information Management Services Directorate is responsible for the information technology needed to maintain and carry out improvements of NIHB's MT data systems. Financial management stewardship falls under the Chief Financial Officer Branch (CFOB), which ensures that appropriate measures are taken to maintain an effective system of internal controls and that frameworks, policies and processes are in place to ensure the effectiveness of departmental financial management. Specific to MT, CFOB will provide leadership and oversight on the proper application and requirements in regards to the stewardship as it relates to financial systems. In summary, the challenges of data collection require the involvement of all three branches to find solutions. Therefore, a new recommendation is addressed below to all three branches.

In conclusion, NIHB has identified some performance measures such as adherence to budgets and service delivery measures in the regions. However the Program continues to experience challenges in collecting reliable and timely data to identify and monitor operational performance measures that would guide progress in MT benefit delivery.

Recommendation 1

It is recommended that the Assistant Deputy Ministers of the First Nations and Inuit Health Branch ensure that the Non-Insured Health Benefits Program establish additional medical transportation operational performance measures and monitor performance data to guide the progress of the medical transportation benefit towards its objectives.

Management response

Recommendation 2

It is recommended that the assistant deputy ministers of the First Nations and Inuit Health Branch, in collaboration with the Assistant Deputy Minister of the Corporate Services Branch and the Chief Financial Officer, work towards a strategy, with a budget and project timelines, to address the challenges in data collection for both operationally managed benefits and those delivered through contribution agreements, in order to improve the monitoring of performance.

Management response

3. Risk management

3.1. Risk management

Audit criterion: Management identifies, assesses, and responds to program risks.

The Program uses the departmental risk assessment tools available on the departmental Intranet, such as the template to create the program risk assessment.

The current MT risk assessment is relatively recent. The issues identified in the 2010 observation on risk management have been resolved. NIHB's headquarters staff led the exercise of re-examining risks to consider external and inter-governmental risk with regional input. The process began in October 2010 and was finalized on April 1, 2011. The draft assessment was then vetted through the NIHB Risk Management Committee, the NIHB Managers' Forum, and the NIHB-HQ Directors Meeting. NIHB plans to review the MT risk assessment on an annual basis to re-evaluate risk levels, identify new or foreseeable risks, and/or remove risks that are no longer applicable to the MT benefit.

The NIHB Manager's Forum has a broad mandate (see Section B2.2 of this report) which includes the assessment of risks and the development of mitigation strategies to ensure that the Program is managed in an efficient manner. The Forum provides an important medium through which senior program staff in the regions and at headquarters can bring forth foreseeable and/or developing risks to all regions. Risk issues were discussed at the June 2012 meeting.

The NIHB Risk Management Committee meets quarterly and deals with both overarching risk to the NIHB Program such as disaster management and demand growth, and risks associated with the NIHB Program's individual benefits, including MT, such as the reduction of transportation service providers.

In conclusion, the organization has a proper process to identify, assess and respond to program risks.

4. Internal control

4.1. Information systems

Audit criterion: Information systems are adequate for the control of operational processing.

As a follow-up to the 2010 audit, and in response to identified weaknesses in some of the MT information systems, specific controls in the MTRS and the OMTS and in the processes and procedures surrounding the systems were examined. The work done as part of this audit does not constitute a full information technology audit of these systems. The areas addressed were limited to: transaction processing; general computer controls (user access and access to specific business functions only); logical access controls and segregation of duties; operational risk management; and change management (OMTS only). The results of the information application control testing are presented in Appendix C.

As mentioned in the background section of this report, Alberta's Accurate Financial XChange Ltd. (AFXL) system was not part of the scope of this audit and therefore was not tested at this time. Controls over the MT information systems are important because they contain the records of operationally managed MT activities for each region, such as the warrants that can be exchanged by the bearer for travel services (flights, accommodations) and payment batches that can be uploaded to SAP to generate payments.

MTRS controls

The MTRS is used by the Atlantic, Quebec, Manitoba, Saskatchewan, British Columbia and Yukon regional offices of Health Canada and by some First Nations to record the details of medical transportation benefits. Some regions, for example, Manitoba, use the system "live" to approve operationally managed travel. Other regions and First Nation bands use the system after the fact to submit detailed information on approved benefits. Version 5.1 of MTRS, released in November 2011, included a module for export of payment batches to SAP but as of July 2012, only the Quebec region was using the payment module.

The use of the new MTRS payment module for payments to suppliers is of concern. [sensitive security-related information removed in accordance with the exemption provisions of the Access to Information Act] Currently, the risk of inappropriate adjustments is low because only the Quebec region staff use the payment module. If other regions adopt this feature, the number of Health Canada staff who would have access to the payment module would remain limited but the potential for errors or abuse would increase. Without proper controls, the risk is that manual adjustments to the payment file could go undetected.

Additionally, with the introduction of the MTRS payment module, there should be a consistent process established across the regions for tracking and resolving exception errors in MTRS batches rejected by SAP.

[sensitive security-related information removed in accordance with the exemption provisions of the Access to Information Act]

OMTS controls

The OMTS system is used "live" daily in three Ontario zone offices to document the approval of trips and warrants, the confirmation of attendance at appointments and to process the invoices of travel suppliers through the export of payment batches to SAP. OMTS is managed by NIHB in the Ontario Region and IMSD has no role in its operations, maintenance or development.

Following a recommendation from the 2010 Audit, application controls have been improved so that the batch payment file is transmitted from within the OMTS directly to SAP. Until identified in this audit, these same controls had not been added to protect batch payment files in other NIHB Ontario systems (for example, vision and mental health). They are now in place for NIHB's Ontario vision and mental health systems. Revisions may also be necessary in other regions that have adopted the NIHB Ontario vision system. However, since the observations concerning NIHB Ontario's vision and mental-health systems are beyond the scope of this audit, they were conveyed to NIHB but were not further investigated. The Ontario region has an informal process to track and resolve any errors in batch transfers (control exceptions process) from OMTS to SAP. However, formalizing this process would help to ensure that errors are resolved in a consistent manner.

The OMTS follows central management's best practices of user accounts, unique user ID's linked to specific user roles, review of user privileges in the course of annual user accounts renewal and comprehensive access logs that identify who makes changes to records in the system. The regions could benefit from a User Account management framework to clearly document roles, responsibilities, risks and controls. A user account management process has yet to be developed and documented.

In the processing of medical transportation benefits, there should be controls to identify and prevent incompatible tasks and to introduce compensating controls where strict segregation of duties is not possible. Ontario NIHB management has not prepared a formal segregation of duties matrix to identify incompatible roles within the OMTS, SAP and signing authorities. The Ontario region has documentation on OMTS, SAP and Section 34 functions but the Program would benefit from a formal segregation of duties approach across all of its zones.

[sensitive security-related information removed in accordance with the exemption provisions of the Access to Information Act]

There is no process to periodically review the segregation of duties. For example, in the Sioux Lookout Zone office, staff had the responsibility to both approve warrants and attendance of the appointments on the same trips. The OMTS is not currently designed to segregate these two tasks. If segregation cannot be achieved through system changes, it may be achieved through team re-structuring and periodical verification that staff is not confirming appointments when they have set up the related warrants. This process allows an analyst to both issue warrants and release them for payment by entering false information in the confirmation of attendance.

Operational risk management processes should ensure that security incidents are defined, categorized and dealt with through standard operating procedures that include the escalation of more serious incidents. Unlike the MTRS, the OMTS does not have a formal entity or process to document and deal with incidents. The OMTS incidents are escalated to the NIHB Operations Manager who contacts the respective appropriate IT resource for assistance, be that the national service desk or regional information technology (IT) consultant. Without a formal incident process, there is a risk that incidents will not be identified and resolved.

In conclusion, information systems require improvements to ensure the adequate control of operational processing. As well, improvements are needed in the processes, roles and responsibilities and documentation around the use of the information systems.

Recommendation 3

It is recommended that the Assistant Deputy Ministers of the First Nations and Inuit Health Branch, in collaboration with the Assistant Deputy Minister of the Corporate Services Branch and the Chief Financial Officer, assess the vulnerabilities of the Medical Transportation Records System and the Ontario Medical Transportation System as identified in this audit and develop appropriate measures to mitigate the risks associated with these vulnerabilities.

Management response

OMTS change management

Changes to the OMTS are authorized centrally and tracked in a change request system.  However, best practices also include a documented change management framework or methodology, including system processes and standards to be followed. Tests on specific changes to the OMTS revealed that several steps in the change process are not documented, including impact assessment and analysis, business requirements, system specification or use cases.

Of particular concern, the process to test system changes is not performed in a separate testing environment. All testing of system changes is done directly in the production environment. There is no framework for the development of test plans and test cases to verify the successful implementation of changes and any inadvertent impacts.

Finally, the maintenance and development activities for the OMTS are contracted to an external systems developer by NIHB management in the Ontario region. The sustaining activities are all undertaken by one person and leaving NIHB without sufficient system documentation to allow other programmers to readily understand the system design. Therefore, the permanence of programming knowledge of the OMTS is at risk.

In conclusion, while there are several positive points about the Ontario region's change management process, practice and tools, there are significant deficiencies that could make the OMTS inoperable (obsolete) and threaten the medical transportation operations in the Ontario region.

Recommendation 4

It is recommended that the assistant deputy ministers of the First Nations and Inuit Health Branch ensure that the Ontario region implement a change management framework for the Ontario Medical Transportation System including a separate test environment, a framework for testing and system documentation.

Management response

4.2. Recipient monitoring

Audit criterion: Risk-based monitoring activities are conducted, including the review of recipient reporting, and the receipt of deliverables before the release of payments.

Monitoring of recipient activities under contribution agreements is done at the regional level and includes the tracking of deliverables (recipient reporting), regular correspondence and site visits.

The main tools used in NIHB are:

• the Agreement/Recipient Risk Assessment Tool (ARRAT), used since 2010-11, which provides general guidance on the level of interaction with communities at higher risk.
• the Management of Contracts and Contributions System (MCCS), which is an electronic system that enhances the FNIHB Programs' ability to report, monitor, and audit its contracts and contribution agreements. The MCCS is used to generate contribution agreements, to store correspondence and reports, and to track the overdue deliverables.

In the two offices (Manitoba and the Sioux Lookout Zone) visited during the audit, there was evidence of monitoring activities in files and through interviews. Although the two regions have a good understanding of which communities are experiencing difficulties, the two regions did not have pre-established monitoring or site visit plans. As a result, site visits were undertaken in reaction to specific issues rather than efforts to target recipients based on history, coverage, materiality, capacity issues or other risk measures.

NIHB's Benefit Management Division has developed template reports for site visits. However, these tools were not being utilized in the two offices visited. NIHB's guidelines^{Footnote 1} suggest the creation of a questionnaire in advance of site visits on key topics such as geographical features, MT staff in the community, vehicle and driver details, and meals and accommodations. Not using such tools when interacting with a community reduces the ability to capture and share meaningful information between the programs delivered by FNIHB in that community.

As noted in the 2010 audit, recipients are only required to report their total actual annual expenditures for MT. This does not provide sufficient information to program managers to undertake an assessment of whether the funding was spent on appropriate expenses for MT or to identify root causes for deficits. Most NIHB recipients routinely prepare schedules to their financial statements with details of revenue and expenditures under various programs. Some recipients already provide such a schedule for MT. Requiring a schedule to show the revenues and cost categories (salaries, gas, repairs, insurance, accommodation, overhead etc.) for MT as part of the recipients annual financial statements would provide better information for recipient monitoring without an undue reporting burden for recipients.

In conclusion, except for the observations mentioned above, risk-based monitoring activities were evident, including review of recipient reporting, and receipt of deliverables before the release of payments.

Recommendation 5

It is recommended that the assistant deputy ministers of the First Nations and Inuit Health Branch ensure that the Non-Insured Health Benefits Program revise the contribution agreement template(s) to require detailed annual financial reporting on medical transportation expenditures by cost category.

Management response

Recommendation 6

It is recommended that the assistant deputy ministers of the First Nations and Inuit Health Branch ensure that the Non-Insured Health Benefits Program officers formalize monitoring practices related to medical transportation benefits and use templates developed by Non-Insured Health Benefits headquarters, including site visit reports.

Management response

4.3. Compliance with FNIHB guidelines

Audit criterion: Contribution agreements, including funding levels, follow the guidelines set by NIHB and FNIHB.

Regions use the contribution agreement template developed by FNIHB's Health Funding Agreement Directorate. FNIHB has also launched 'Knowledge in a Book: Business processes for managing FNIHB Contribution Agreements' which outlines financial and management control measures to be put in place by program managers, including the requirements that must be met for payments to be made. The other guideline tool is the NIHB Medical Transportation Contribution Agreement Manual (CA Manual), which describes additional tips related to the management of MT contribution agreements (CAs) and provides interpretations of the NIHB Medical Transportation Policy Framework.

The CA Manual states that it is 'critical that the assessment and rationale for the amount awarded is documented in all circumstances'. In the files reviewed in the Sioux Lookout Zone office, rationales for funding level decisions were not present. Currently, the CA Manual lists three factors that impact on funding levels. However, there is no national formula supporting funding level decisions and thus, there are inconsistencies across the country in determining funding levels.

To support First Nations and Inuit people in reaching an overall health status that is comparable with other Canadians, it is expected that funding for medical transportation for eligible clients provided under contribution agreements be comparable amongst communities. Though both regions visited strive to adhere to FNIHB's CA guidelines, funding levels are based on historic expenditures, reviews of interim and annual financial and program activity reports, as well as routine communication with CA recipients. As a result, similar communities in Ontario and Manitoba could receive different funding.

Although identical funding levels are impracticable, as provincial differences and recipient needs vary, a consistent and formulated approach can be shared between regions and communities with similar needs, through national guidelines.

In conclusion, except for the observation on funding levels and the documentation of rationales, contribution agreements follow the guidelines set by NIHB and FNIHB.

Recommendation 7

It is recommended that the assistant deputy ministers of the First Nations and Inuit Health Branch ensure that a national guideline be developed to assist medical transportation program officers in determining and documenting funding levels for consistency in approach across Canada.

Management response

4.4. Client and service eligibility

Health Services

• Insured health services by provincial/territorial health plans (for example, appointments with physician, hospital care);
• diagnostic tests and medical treatments covered by provincial/territorial health plans;
• alcohol, solvent, drug abuse and detox treatment;
• traditional healers; and
• non-insured health benefits (vision, dental, mental health).

Audit criterion: There is a rigorous approach to the confirmation and approval of client and service eligibility.

For operationally managed benefits, client eligibility is verified through FNIHB's Status Verification System. Notwithstanding the eligibility of the client, medical transportation benefits only apply if the nature of the appointment/treatment falls within the definition of an eligible service.

Medical transportation benefits do not apply if the transportation cost is covered by another insurance plan (or example, motor vehicle insurance, workers compensation, or private/group insurance plans).

Program officers determine the eligibility of the service by carefully reviewing the medical referral, the client's medical transportation records, by asking appropriate questions, and by referring to the Operations Manual. Where the eligibility is not clear, the decision is referred to a supervisor.

Tests on a sample of transactions confirmed that the clients and services were eligible.

In conclusion, there is a rigorous approach to the confirmation and approval of client and service eligibility.

4.5. Payment support

Audit criterion: Medical transportation expenses are adequately supported and approved prior to payment.

For operationally managed benefits, the claims verification process entails the matching of provider invoices against transportation warrants (henceforth referred to as 'warrants') and the verification of client's attendance at the appointment, as recorded in the medical transportation system. A warrant is used to pre-approve eligible MT benefits (for example, transportation, accommodation and meals) for NIHB clients. Warrants include information such as the name of the traveller, the purpose of the trip, the date until which it can be used, the name of the service provider and the associated cost (either negotiated or as per the provider's regular fares). This document demonstrates Health Canada's commitment to pay the provider for the services rendered as detailed on the warrant.

Warrant forms stipulate four conditions governing their use: copies must be remitted to providers; no alterations are permitted; unused warrants must be surrendered and cannot be exchanged for money; and, warrants must be signed by clients. However, warrant forms make no reference to the terms and conditions on how services are to be provided by third-parties and paid for by Health Canada. As a result, the Crown has little recourse in case of disagreements over billing or to further verify that services were provided in accordance with the Program's terms and conditions.

As for contribution agreements, the claims verification process includes confirmation that any required reports have been received. As evidenced by file review, approval by the Regional Director is obtained prior to payment release whenever reports are overdue, which may lead to withholding of payments.

Audit tests were conducted on payments for both operationally managed benefits and those provided through contribution agreements. In the sampled items, MT expenses were adequately supported and approved prior to payment. Verification, review and adjustments, or corrections (where required) were evidenced in files. As per these tests, certifications under sections 33 and 34 of the FAA were adequately conducted.

In conclusion, MT expenses are adequately supported and approved prior to payment. However, reference to terms and conditions on how services are to be provided by providers and paid for by Health Canada require clarity.

Recommendation 8

It is recommended that the assistant deputy ministers of the First Nations and Inuit Health Branch ensure that the Non-Insured Health Benefits Program review transportation warrants making reference to terms and conditions on how services are to be provided by suppliers and paid for by Health Canada.

Management response

C - Conclusion

The audit concludes that:

• the implementation of the management action plan has been effective in addressing most of the recommendations made in the Audit of the Non-Insured Health Benefits Medical Transportation (October 2010); and
• the management control framework for the delivery of medical transportation benefits requires improvements in managing key risks.

Improvements have been noted in the risk management identification and mitigation strategy, security of payment file transfers of the Ontario Medical Transportation System transactions and the claims verification process in the Ontario region. However, there are eight recommendations in this audit, including one that replaces a recommendation from the 2010 Audit of the Non-Insured Health Benefits Medical Transportation.

The Non-Insured Health Benefits' delivery of medical transportation faces two critical challenges that require immediate attention:

• challenges in obtaining complete, accurate and timely nation-wide data; and
• information technology control issues.

Observations were also made with respect to recipient monitoring, consistency in funding approach, and clarity of the terms and conditions of services provided by suppliers and paid for by Health Canada.

Appendix A

Appendix B

Appendix C

Appendix D - Overview of Medical Transportation Data Store sources

Appendix E

Appendix F

Appendix G - Assessment Rating Guide

1. No progress or insignificant progress

No action taken by management or insignificant progress. Actions such as striking a new committee, having meetings, and generating informal plans are insignificant progress.

2. Planning stage

Formal plans for organizational changes have been created and approved by the appropriate level of management (at a sufficiently senior level, usually Executive Committee level or equivalent) with appropriate resources and a reasonable timetable.

3. Preparation for implementation

The entity has begun necessary preparation for implementation, such as hiring or training staff, or developing or acquiring the necessary resources to implement the recommendation.

4. Substantial implementation

Structures and processes are in place and integrated in some parts of the organization, and some achieved results have been identified. The entity has a short-term plan and timetable for full implementation.

5. Full implementation

Structures and processes are operating as intended and are implemented fully in all intended areas of the organization.

6. Cancelled

Audit recommendations that are deemed to be obsolete or have been superseded by another recommendation.