Back to Table of Contents
The findings and conclusions contained in this report are based on sufficient and appropriate evidence gathered in accordance with procedures that meet the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing . The advisory engagement thus conforms to the Institute of Internal Auditors’ International Standards for the Professional Practice of Internal Auditing as supported by the results of the quality assurance and improvement program. The opinions expressed in this report are based on conditions as they existed at the time of the advisory engagement and apply only to the consulted entity.
Back to Table of Contents
Canadian Forces Health Services (CFHS) provides health care to Canada's Regular and Reserve Force personnel when and as required. The Department of National Defence (DND) reported spending of $721 million on Military Health Care in the Departmental Results Report for Fiscal Year (FY) 2016/17.
Canadian Armed Forces (CAF) members are generally not covered under provincial health plans in accordance with the Canada Health Act. Instead, members receive non-emergency outpatient health services through CFHS health clinics staffed by military and DND-employed health care providers. When CAF members require after-hours or more specialized health services, they are referred to civilian health care facilities.
Civilian health service providers submit their claims for payment for services provided to CAF members through the Federal Health Claims Processing Service (FHCPS), administered by Medavie Blue Cross (MBC) under contract.
DND, the Royal Canadian Mounted Police (RCMP) and Veterans Affairs Canada (VAC) are partners in the FHCPS contract. The Director Health Services Delivery within CFHS manages this contract on behalf of DND/CAF.
MBC pays eligible claims to civilian health service providers and sends paid claims for reimbursement. DND receives claims through FHCPS for the services received by CAF members and pays the amounts owed. DND paid $144 million for medical services related to the FHCPS contract in FY 2016/17.
Back to Table of Contents
Over the FYs 2010/11 to 2016/17, the number of medical services provided and the number of members treated has declined slightly. However, the total spending and the average transaction value has increased.
The top three programs of care in FY 2016/17 consisted of “Hospital Services” ($60 million ), “Medical Services” ($30 million ), and “Related Health Services” ($20 million).
Back to Table of Contents
Under Canada’s defence policy: Strong, Secure, Engaged , the number of Regular Force and Reserve Force personnel is expected to increase by 3,500 (to 71,500) and 1,500 (to 30,000) respectively. With the policy’s focus on “Total Health and Care for our People,” the demand for health services is expected to increase.
CFHS recognized the potential to improve the delivery of health services by increasing the use of data received from the FHCPS contract for management decision making.
In 2017, CFHS approached the Assistant Deputy Minister (Review Services) (ADM(RS)) to seek advice on how to improve its data analytics capability while also addressing management action plans from the 2014 ADM(RS) Audit of Canadian Forces Health Services.
ADM(RS) initiated this advisory engagement to provide assistance to CFHS related to the FHCPS contract, including:
advice on improving the use of information for decision making;
applying selected data analytics (Annex A ); and
facilitating a brainstorming session to explore uses of data analytics.
Concurrently, ADM(RS) conducted a follow-up on the 2014 Audit of Canadian Forces Health Services – tabled as a separate report.
Back to Table of Contents
Data analytics refers to the process of inspecting, cleansing, transforming and modeling data in order to discover useful information that provides value as a decision-making tool.
The application of data analytics to the field of health services allows for the examination of patterns within data in order to identify opportunities for improving the quality and efficiency of health care.
Additional benefits of data analytics include the following:
Provides deeper and broader monitoring coverage;
Improves the timeliness and responsiveness of monitoring;
Provides a less costly, continuous monitoring mechanism; and
Provides regular assessments of data quality and integrity. Issues related to data quality and integrity usually reflect an internal control issue and can be reviewed accordingly.
Back to Table of Contents
Objective
To provide advice to CFHS on improving the process of receiving, validating and effectively analyzing the information from the FHCPS contract in support of decision making in the delivery of health services.
Approach
The following approaches were used to assess governance structures, reporting and information requirements, capacity to analyze information and potential to improve:
Interviews with CFHS personnel and contractor staff;
Review and analysis of documents related to managing and administering the FHCPS contract, including contract documents, contractor-provided reports, minutes to meetings, etc.;
Selected data analytics undertaken by ADM(RS) to illustrate examples of the types of analysis possible using available FHCPS data (Annex A ); and
Facilitation of a brainstorming workshop to explore the information requirements that may support decision making in the delivery of health services.
Scope
Activities related to the FHCPS contract by CFHS from April 1, 2016 to September 30, 2017.
Data analytics was applied to data from April 1, 2010 to March 31, 2017.
Back to Table of Contents
Governance and Accountability Structures – Do the current governance and accountability structures within CFHS facilitate the flow of information between internal and external stakeholders for the FHCPS contract?Information Requirements** – Are there mechanisms to confirm that the information received through the FHCPS contract is relevant, sufficient, complete and timely to support management decision making?Reporting Requirements – Are processes established to ensure that the reporting requirements within the terms of the FHCPS contract and the memorandum of understanding with VAC are respected?Capacity to Analyze Information – Does CFHS have the capacity to process and analyze the information received through the FHCPS contract to inform evidence-based management decisions in the health services program?Improvements to Current Processes** – Can improvements be made to the processes for receiving, validating and analyzing information under the FHCPS contract in support of decision making in the delivery of health services?
* As agreed upon with management.Annex A ) were performed to support findings on information requirements.
Back to Table of Contents
The capability for data analytics within CFHS is generally limited and ad hoc.
Opportunities exist to develop the capacity to leverage the use of existing data to improve evidence-based decision making.
Table 2. Summary of Results by Area Examined. This table lists the areas examined, the assessments and the opportunities for improvement.
Area
Assessment
Summary of opportunities for improvement
Governance and Accountability Structures
Minor improvement possible
Define responsibilities of internal stakeholders for the identification and communication of key performance questions.
Establish periodic consultations with stakeholders to solicit information requirements.
Information Requirements
Moderate improvement possible
Identify information requirements to support management decision making.
Improve data quality by identifying high-risk transactions for further review.
Reporting Requirements
Minor improvement possible
Introduce additional contract performance measures and service standards.
Increase the use of existing contract provisions.
Capacity to Analyze Information
Significant improvement possible
Improve guidance to stakeholders on the use of data analytics.
Develop data analytics capability by training staff or acquiring external expertise.
Improvement to Current Processes
Significant improvement possible
Formalize the use of data fields and coding information.
Identify, establish and update data for key performance measures.
Target compliance verifications and strengthen control weaknesses.
Back to Table of Contents
Do the current governance and accountability structures within CFHS facilitate the flow of information between internal and external stakeholders for the FHCPS contract?
The governance structure, accountabilities and responsibilities are clearly established and well defined between CFHS and its external stakeholders – MBC, VAC, and the RCMP.
The four parties meet on a scheduled and regular basis, which facilitates the flow of information.
The Director Health Services Delivery is focused on processing payments and handling ad hoc requests from other internal stakeholders such as CAF clinics, medical practitioners and other subject matter experts.
Opportunities for improvement
The roles and responsibilities of internal stakeholders could be better defined to identify and communicate key performance questions to support evidence-based decision making.
Periodic consultations with internal stakeholders could be conducted to identify information required to support key performance questions.
Benefits of improvement
Facilitating collaboration between CFHS stakeholders with responsibilities for policy development, program management, and service delivery to identify key performance questions.
Improving the understanding of how program decisions may be supported using the billing data that is provided under the FHCPS contract.
Identifying opportunities to integrate data from other systems such as the Canadian Forces Health Information System (system that contains the electronic health record for CAF members), with data from the FHCPS contract.
Articulating business rules, process flows, service standards, or other parameters to identify trends and anomalies.
Identifying relationships within existing data to assess the health care costs and impacts of policy decisions, such as:
Gender Based Analysis Plus (i.e. impact of increasing representation of women in CAF);
increase in special forces members, who have higher health services costs;
changes to retirement age;
increase in the number of concurrent missions; and
predicting demand for specialists or services.
Back to Table of Contents
Are there mechanisms to confirm that the information received through the FHCPS contract is relevant, sufficient, complete and timely to support management decision making?
CFHS management relies on standardized and ad hoc reports provided by the FHCPS contractor.
Opportunities for improvement
Director Health Services Delivery could use data analytics to improve data quality and provide management with better information for decision making.
Opportunities exist to explore trends and patterns in data, including the use of management dashboards.
Some examples of data analytics applications include the following:
Risk Profiling: identifying transactions with potential quality issues for further review.
Resource Optimization: comparing the cost of external providers paid on a per visit basis with the cost of an in-house provider; analyzing the variances in costs for a given medical procedure charged by different providers.
Annex A provides further examples of the information that could be generated through data analytics.
Benefits of improvement
Increasing cost effectiveness and optimizing the delivery of health services, using trends and patterns to predict changes in demand for types of services.
Identifying potentially erroneous transactions (such as miscoded transactions, duplicate transactions and incompatible services) for further investigation.
Improving control measures and monitoring, such as to focus monitoring resources on high-risk areas.
Analyzing cost variations by region, service provider, or service to inform decisions regarding rate limits and premiums.
Exploring the costs of different service delivery models, such as using in-house CAF medical practitioners, contracted medical practitioners, or civilian health service providers in the public health system.
Back to Table of Contents
Are processes established to ensure that the reporting requirements within the terms of the FHCPS contract and the memorandum of understanding with VAC are respected?
The reporting requirements as set out in the FHCPS contract have been met.
The governance structure and responsibilities of participating departments in the collaborative management of the FHCPS contract are being respected, as described in the memorandum of understanding between DND/CAF and VAC.
There is a process in place to obtain pre-defined and ad hoc data and reports from MBC.
Queries to the FHCPS database sometimes require modifications to existing report templates. DND/CAF, VAC, and the RCMP initiate Task Authorizations with MBC for these change requests. These requests are then prioritized by the three partner departments for action by MBC.
CFHS does not currently make effective use of provisions in the contract that allow for additional data, including data analytics services that can be requested from MBC.
Opportunities for improvement
Establishing additional performance measures and service standards over the management of the FHCPS contract, such as the amount of time it takes to fulfill Task Authorizations from MBC, would improve the prioritization of information requests to ensure that they are processed in a timely manner.
As CFHS develops additional information requirements, requests could be made to MBC to better make use of existing provisions of the contract, such as defining the criteria for claims verification and audits of service providers.
Benefits of improvement
Setting the conditions to facilitate data analytics capability by establishing a structure to obtain required information on a timely basis.
Maximizing the value for money received from the FHCPS contract by leveraging contractor capacity to strengthen control measures and monitoring, and to provide data analytics support.
Back to Table of Contents
Does CFHS have the capacity to process and analyze the information received through the FHCPS contract to inform evidence-based management decisions in the health services program?
The capability for data analytics within CFHS is generally limited and ad hoc.
Two national coordinators within the Director Health Services Delivery organization are responsible for generating standard queries and pre-defined reports.
Subject matter experts identify data and reporting requirements on an ad hoc basis.
Information is either extracted from standard reports currently available, or customized reports are requested from MBC.
Management has expressed an interest in developing data analytics capabilities within Director Health Services Delivery, although there is currently no plan in place to develop them.
Opportunities for improvement
Use of existing data could be strengthened to support management decisions. This could be achieved by providing guidance to headquarters personnel and subject matter experts in medical practice, information systems and financial management on the use of data analytics for decision making, and by making a more coordinated effort to develop data and reporting requirements.
Increase the capacity to perform advanced data analytics. This would require trained staff and the acquisition of data analytics or business intelligence software. This capability could be developed within CFHS, obtained from MBC, or outsourced.
Benefits of improvement
Improving the effectiveness and efficiency of health services delivery based on improved business intelligence. For example:
Monitoring key cost factors such as fee limits, preauthorization requirements;
Prioritizing improvement projects and informing service delivery options;
Informing clinical management and utilization decisions (i.e. benchmarking clinical best practices or targeting field reviews where long-term care is provided without second examinations); and
Analyzing cost variations to inform decisions regarding controlling fees.
Back to Table of Contents
Can improvements be made to the processes for receiving, validating and analyzing information under the FHCPS contract in support of decision making in the delivery of health services?
Improvements could be made in the following areas:
Receiving and validating data:
Collaborate with FHCPS partners and contractors to identify, strengthen, and implement necessary information technology controls.
Improve the administration of the FHCPS contract by providing continuous feedback on information requirements from internal stakeholders such as clinical administrators or medical practitioners.
Require and enforce the proper use of data fields and coding information by health care providers to increase the accuracy in the characterization of transactions.
Processing data:
Conduct regular consultations with CFHS subject matter experts to identify and update performance measures of interest.
Select appropriate performance indicators to guide evidence-based business decisions.
Analyzing data and leveraging information for business intelligence:
Refine the process for risk profiling to target compliance verification.
Identify and remediate control weaknesses in the management of the FHCPS contract.
Provide timely data to management and subject matter experts to improve service delivery and efficiency in the use of resources.
Back to Table of Contents
CFHS could improve the effectiveness and efficiency of the management of health services within the FHCPS contract through the development of data analytics capabilities to leverage existing information by:
Increasing the engagement of internal stakeholders as part of the governance structure to identify information requirements;
Establishing performance measures based on business requirements and key metrics;
Acquiring or developing the capability to execute data analytics; and
Using the resulting information for decision making.
Development of such a capability would ultimately lead to better health care for CAF members, and significant cost savings.
Back to Table of Contents
As Military Personnel Command (MPC) develops its plan for the use of data analytics, it should include a capability within CFHS for using data analytics in managing health services, taking into account the following:
Developing a capability to leverage the use of data analytics;
Establishing information requirements with stakeholders and obtaining the data needed;
Making effective use of the capabilities available through existing contracts;
Identifying the capability gap needed to meet requirements; and
Building a strategy to fill this gap.
CFHS, within MPC, should pursue opportunities to strengthen internal control measures, such as those identified in this advisory report, in order to improve the quality of care, and to achieve cost savings. This plan should include, but should not be limited to the following:
Transactions that may be miscoded;
Possible duplicated payments;
Fees paid in excess of limits; and
Improving utilization of resources.
Back to Table of Contents
ADM(RS) uses recommendation significance criteria as follows:
Very High —Controls are not in place. Important issues have been identified and will have a significant negative impact on operations.High —Controls are inadequate. Important issues are identified that could negatively impact the achievement of program/operational objectives.Moderate —Controls are in place but are not being sufficiently complied with. Issues are identified that could negatively impact the efficiency and effectiveness of operations.Low —Controls are in place but the level of compliance varies.Very Low —Controls are in place with no level of variance.
Management Action
ADM(RS) has developed an analytical dashboard for the FHCPS contract, which CFHS intends to implement once received. This will require a transition period, where ADM(RS) provides the analytical tool along with the necessary training that will allow CFHS to manage and further develop it over time. CFHS intends to leverage the chain of command and existing lines of communications to share and further progress the dashboard. Specifically, CFHS (Directorate of Health Service Delivery) will progress the data analytics dashboard through the collaborative efforts of Clinic Managers, Base Surgeons, and administrative support staff. CFHS internal initiatives, such as Modernization (Directorate of Quality and Performance) and Digital Health Environment are expected to be key enablers in successfully developing a data analytics capability. CFHS requires guidance and direction from MPC’s Chief Data Officer and Information Management Officer before proceeding beyond a dashboard analytics tool.
OPI: MPC/CFHSTarget Date: July 2020
Management Action
Information requirements for a decision-making tool were identified in an ADM(RS)-facilitated analytical workshop. The ADM(RS) Advisory Report identified gaps in information requirements in selected areas, which will be adopted and further developed as understanding, capacity, and capability of data analytics within CFHS develops. CFHS (Director Health Services Delivery) will conduct quarterly workshops with key stakeholders (e.g. Director Medical Policy, Directorate of Quality and Performance, Director Force Health Protection, Director Mental Health, and chain of command) in order to identify additional information requirements, high-risk areas, and further develop an analytics capability. Key indicators will not be limited to financial or contractual requirements, and will incorporate aspects of clinical governance. CFHS intends to develop an ability to identify the health-related costs to changes in policies and spectrum of care (e.g. change of compulsory retirement age from 55 to 60, and increasing CAF’s female demographic). CFHS requires guidance and direction from MPC.
OPI: MPC/CFHSTarget Date: July 2020
Management Action
As CFHS’ data analytics capability develops, its ability to direct Medavie Blue Cross (MBC) to conduct targeted analysis will mature. In time, CFHS will better influence what MBC analyzes in its quality assurance program, specifically intended to prevent: double-billing, de-bundling of services, incorrect codes being used, payment for unentitled services (e.g. outside spectrum of care), surpassing limits (in terms of cost and/or frequency). In addition, CFHS intends to leverage MBC’s audit program services on an ad hoc basis – when it makes sense financially and/or HR-wise to do so. As CFHS’ data analytics capability matures, its ability to develop pre-defined queries or static reports will progress. CFHS intends to incorporate this analysis in future contract negotiations.
OPI: MPC/CFHSTarget Date: July 2020
Management Action
CFHS intends to compare the requirements in MAP 1.2 to the existing capabilities of MAP 1.3 in order to identify gaps. As CFHS develops its data analytics capability, such gaps will inherently be addressed or mitigated via MAP 1.2 and MAP 1.3.
OPI: MPC/CFHSTarget Date: July 2020
Management Action
As CFHS develops its data analytics capability, its ability to develop a strategy to address the gap identified in MAP 1.4 will become more evidenced. Its ability to do so is dependent on the guidance and direction it receives from MPC, resources available, the implementation of Modernization, Digital Health Environment, and the analysis from MAP 1.1, 1.2, 1.3, and 1.4.
OPI: MPC/CFHSTarget Date: July 2020
Management Action
CFHS collects, collates, and analyzes FHCPS’ expenditures to support integral decision-making capabilities. The ADM(RS) Advisory Report identified some potential high-risk areas that have been mitigated, and it also validated the efficacy of the financial data collected, collated, and analyzed on the FHCPS contract since FY 2010/11. MAP 1.1, 1.2, 1.3, 1.4, and 1.5 will identify other high-risk areas that will be addressed and/or monitored. On a regular basis, MBC will report on identified, or potential, high-risk areas to the FHCPS National Manager, which will be incorporated in the data analytics dashboard provided by ADM(RS). This will be used to mitigate and/or prevent future errors and provide leadership with improved data analytics. The first quarterly review will occur by the end of September 2018.
Actions to address the specific areas identified include the following:
The 64 cases of miscoded transactions were by the same provider, who has been reminded of the correct codes to use going forward.
Medavie Blue Cross is amending the IMS query data extract to include fields from the data dictionary that will allow FHCPS clerks to validate possible duplicate payments more precisely and obtain additional details on expenses claimed.
Medavie Blue Cross notifies CAF’s FHCPS team when requests to exceed limits are requested. Each request will be adjudicated on a case-by-case basis.
Quarterly reviews of high-risk areas are now scheduled.
Review by ADM(Fin) on FHCPS’ Section 34 financial framework is in progress, which will include consideration of appropriate controls.
OPI: MPC/CFHSTarget Date: July 2020
Back to Table of Contents
Background
Military Personnel Command requested the assistance of ADM(RS) in order to analyze the information currently received under the FHCPS contract.
The MBC billing data from FYs 2010/11 to 2016/17 was analyzed as follows:
Identify some potential risk areas.
Compare provincial health care rates to rates paid under the FHCPS contract:
Comparison of FHCPS fees paid in Ontario to the Ontario provincial fee guide.
Statistical analysis of variance in FHCPS fees.
1. Potential Risk Areas
ADM(RS) tested the data for the following illustrative risk areas*:
Possible duplicate transactions
Possible miscoded transactions
Fees paid in excess of maximum amount allowed
Transactions for members aged 60+
* Subsequent to communication of the findings, CFHS has initiated investigations into identified risk transactions.
1A. Possible Duplicate Transactions
ADM(RS) tested transactions that involved the same member, receiving the same service (code), from the same provider, on the same day, and billed for the same amount*. These transactions require further investigation by CFHS.
Table 3. Possible Duplicate Transactions. This table summarizes the potential number and cost of duplicated transactions by FY.
Fiscal Year
Potential Duplicate Transactions #
Potential Duplicate Transactions $
2010/11
11,349
$1,645,115
2011/12
16,529
$2,225,099
2012/13
14,162
$1,797,477
2013/14
14,897
$1,857,219
2014/15
14,238
$1,744,411
2015/16
13,310
$2,019,586
2016/17
15,507
$2,453,422
Total 99,992 $13,742,329
* Due to a system limitation, transactions over $10,000 may be split into multiple transactions. These transactions were excluded from the identification of potentially duplicated transactions.
ADM(RS) tested the database to identify cases of possible miscoded transactions. These transactions require further investigation by CFHS:
4,445 transactions for services related to the opposite sex, totalling $550,000 over seven years.
64 instances of male service members billed for caesarean section procedures.*
4,305 transactions of male service members billed for services related to “obstetrical care” totalling $336,000 over seven years.
* These transactions were subsequently reviewed by CFHS and determined to be provider coding errors for other surgical procedures.
1C. Fees Paid in Excess of Maximum Amount Allowed
On August 1, 2015, a dollar or frequency limit was placed on a number of benefit codes. Additional authorizations from CAF medical practitioners were needed if these limits were exceeded. However, FHCPS administrators within DND/CAF did not have the means to easily reconcile these authorizations as there were no identifiers in the system to distinguish these transactions from non-authorized transactions.
Testing of the FHCPS database since the limit was implemented identified:
7,404 transactions that exceeded the dollar limit placed on the medical procedure performed. These transactions represented a total amount of $367,000 in excess of the maximum amount allowed.
5,186 transactions where the submitted amount was higher than the maximum amount allowed, and was reduced by MBC to the maximum.
1,578 transactions where the submitted amount was reduced by MBC to an amount less than the maximum amount allowed.
These results demonstrate possible inconsistencies in the application of maximum fees, and opportunities to strengthen the information technology controls related to transactions exceeding the maximum amounts allowed (i.e. authorization and review).
1D. Transactions for Members Aged 60+
Per ADM(HR-Mil)* Instruction 14/04, the Canadian Forces (Regular Force and Primary Reserve) compulsory retirement age is 60. ADM(RS) analyzed the data for health services provided to members over the age of 60.
Analysis of the database identified 4,565 instances of services provided to 304 members that were aged 60+. The cumulative total of the services provided was $700,000.
Under most circumstances, these services should have been paid for by VAC.
These transactions are being reviewed by CFHS.
* Military Personnel Command was previously known as ADM(HR-Mil).
2A. Comparison of FHCPS Costs to Ontario Fee Guide
ADM(RS) compared the fees paid for services through FHCPS in FY 2016/17 to the Ontario Schedule of Benefits fee guide. As a reference, the Ontario Medical Association (OMA) recommends a 2.22x multiplier for Uninsured Services:
There are over 18,000 different billing codes in the CFHS database. In FY 2016/17, 2,346 benefit codes from the Ontario Schedule of Benefits fee guide were found in the CFHS database. ADM(RS) examined transactions for 5 of the most used Ontario benefit codes and 5 of the Ontario benefit codes that accounted for the largest dollar amounts. The sample represented approximately 9 percent of spending and line items of Ontario benefit codes:
Spending in 2016/17 per occurrence of these sampled benefit codes totalled $1,250,000. This was $733,000 more than if the Ontario rates had been used, and $103,000 more than if 2.22x the Ontario rate had been applied.
Approximately 90 percent of the transactions sampled were at least 2x the Ontario rate.
This information could be used to inform future decisions related to preventive information technology controls, monitoring and/or limiting rates.
2B. Analysis of FHCPS fees
ADM(RS) applied statistical analysis to determine the differences in fees by service and province/territory.
Results indicate a wide variance in fees paid between and within provinces/territories as demonstrated in the seven randomly selected benefit codes shown in Table 4.
