Evaluation of Health Canada’s First Nations and Inuit Health Branch’s Environmental Public Health Program 2010-11 to 2014-15

Download the alternative format
(PDF format, 456 KB, 59 pages)

Organization: Health Canada

Date published: 2017-02-24

Prepared by

Office of Audit and Evaluation

Health Canada and the Public Health Agency of Canada

September 15, 2016

List of Acronyms

CA
Contribution Agreement
CBWM
Community Based Drinking Water Quality Monitor
CFIA
Canadian Food Inspection Agency
CMHC
Canadian Mortgage and Housing Corporation
DPR
Departmental Performance Report
DWA
Drinking Water Advisory
EC
Environment Canada
EH-ER
Environmental Health and Environmental Research
EHIS
Environmental Health Information System
EHO
Environmental Health Officer
EPH
Environmental Public Health
EPHD
Environmental Public Health Division
EPHP
Environmental Public Health Program
FNECP
First Nations Environmental Contaminants Program
FNFNES
First Nations Food, Nutrition and Environment Study
FNEHIN
First Nations Environmental Health Innovation Network
FNHA
First Nations Health Authority
FNIHB
First Nations and Inuit Health Branch
FNWWAP
First Nations Water and Wastewater Action Plan
Gs&Cs
Grants and Contributions
GCDWQ
Guidelines for Canadian Drinking Water Quality
HQ
Headquarters
INAC
Indigenous and Northern Affairs Canada
O&M
Operations and Maintenance
PAA
Program Alignment Architecture
PHAC
Public Health Agency of Canada
REHM
Regional Environmental Health Managers

Table of Contents

List of Tables

Executive Summary

This evaluation provides an update on the relevance and performance of Health Canada's Environmental Public Health Program (EPHP), which was first evaluated in 2011 under Health Canada's Environmental Health and Environmental Research (EH-ER) Cluster Evaluation.

Evaluation Purpose and Scope

The purpose of this evaluation is to assess changes in the performance of Health Canada's EPHP during the past five fiscal years (2010-11 to 2014-15), and to fulfill the requirements of the Financial Administration Act and the Treasury Board of Canada's Policy on Evaluation (2009). The scope of this evaluation included most activities performed by the EPHP, as well as progress made on activities covered under the First Nations Water and Wastewater Action Plan. Not included within the scope of this evaluation were the activities that are now covered under the British Columbia Tripartite Framework Agreement on First Nation Health Governance, which came into full effect in 2014-15, and activities included under recently evaluated horizontal initiatives, namely the Clean Air Agenda and the Action Plan to Protect Human Health from Environmental Contaminants. Furthermore, the evaluation did not assess FNIHB activities related to Inuit and Inuit communities given that those activities were assessed as part of existing external/horizontal evaluation (INAC's Northern Contaminants Program and Environment and Climate Change Canada's Clean Air Agenda).

Program Description

The primary objective of the EPHP is to "identify, address, and/or prevent human health risks to First Nations and Inuit communities associated with exposure to hazards within the natural and built environments." Environmental public health surveillance and risk analysis programming includes community-based and participatory research on trends and impacts of environmental factors, such as chemical contaminants and climate change on the determinants of health (e.g., biophysical, social, cultural, and spiritual).Footnote 1 This programming is directed to First Nations communities south of the 60th parallel, and in the case of the Northern Contaminants Program and the Climate Change and Health Adaptation Program, also to Inuit and First Nations north of the 60th parallel.Footnote 2

Within Health Canada, the EPHP is coordinated regionally by Environmental Public Health Services (EPHS) and supported nationally by the Environmental Public Health Division (EPHD). EPH programming is delivered in First Nations communities south of 60º by Environmental Health Officers (EHOs) employed by Health Canada or First Nation communities and/or Tribal Councils in accordance with the National Framework for the Environmental Public Health Program in First Nations Communities South of 60º. Key programming includes environmental public health assessments; training; and, public awareness activities that focus on eight core areas: Drinking Water; Wastewater; Solid Waste Disposal; Food Safety; Housing; Facilities inspections; Environmental Communicable Disease Control; and, Emergency Preparedness and Response.Endnote i

Between April 1, 2010 and March 31, 2015, the EPHP expended approximately $162 million through set contribution funding agreements and direct departmental spending to identify, assess and address human health risks associated with exposure to hazards within the natural and built environments. One key program difference as of 2014 is that activities are now performed in the province of British Columbia by the First Nations Health Authority (FNHA) as a result of the British Columbia Tripartite Framework Agreement on First Nation Health Governance. Another difference as of 2011 is that the former cluster structure of two groups of programming (Environmental Health and Environmental Research) have now been integrated into one program, namely the Environmental Public Health Program at FNIHB.

Conclusions - Relevance

Continued Need

EPH programming is globally recognized as an essential component of public health systems, and is a key contributor to improving the health and well-being of populations in a holistic manner. The EPHP supports a continued and growing need among many First Nations communities to identify and address human health risks associated with exposure to hazards within the natural and built environments. As identified in the previous evaluation, many First Nations communities continue to experience significant EPH risks compared with other Canadian communities. Ongoing challenges result from issues such as: sub-standard housing and living conditions; poor drinking water quality and poorly operated wastewater systems; a lack of certified water plant operators; climate change; local geography characteristics; and, numerous socio-economic inequalities. While the resolution of EPH issues often falls outside the scope of Health Canada's control, there continues to be a strong, demonstrated need for a program like EPHP to influence health promotion and disease prevention outcomes in First Nations communities. As First Nations communities continue to grow and experience more complex environmental public health risks, the need for EPH assessments, training, public awareness, research and surveillance is expected to increase.

Alignment with Government Priorities

The EPHP continues to align with federal priorities and strategic outcomes sought to improve the health status of First Nations and Inuit communities and individuals.

Alignment with Federal Roles and Responsibilities

Since the previous evaluation, Bill S-8 was passed and the Safe Drinking Water for First Nations Act came into force in 2013. The introduction of this Act moves the federal role in EPH risk mitigation beyond advisory to include a possible regulatory role. During this evaluation time period, safe drinking water regulations for First Nations were under development; however, as of 2016, the Act became subject to review by INAC Ministerial direction, and regulatory development was put on hold. Health Canada continues, therefore, to play an advisory role which varies in each region in order to meet specific community needs and priorities.

As different federal departments have a role to play in supporting First Nations communities in addressing EPH risks, it is necessary that the contribution of all federal departments is coordinated and aligned to maximize effectiveness and achieve common strategic outcomes. Currently, the Health Canada role in drinking water remains advisory, and the activities delivered under the EPHP align with this advisory role. Health Canada also does not currently have the mandate to provide funding to communities to address or remediate risks, as this is the purview of INAC. While there is no duplication, there are opportunities for greater coordination and integration at the federal level.

Conclusions - Performance

Achievement of Expected Outcomes (Effectiveness)

Building on the achievements identified in the previous evaluation, over the past five years the EPHP has demonstrated clear progress in achieving its immediate outcomes. For example, findings indicate that data collection is more timely and findings are disseminated at various levels either through recommendations or other types of reporting; there is an increased use of evidence to guide EPH activities with relevant research being conducted in communities to address EPH priorities; and an awareness of EPH risks among First Nations communities has increased overall, which has further contributed to an increased use of, and demand for, EPHP services in communities.

Similar to the previous evaluation findings, while progress on immediate outcomes is evident, the extent to which intermediate outcomes have been achieved is less clear. There are three general steps required to mitigate EPH risks that include activities that identify, then assess, and then address issues (if required from the assessment).

Findings from this evaluation indicate that the EPHP has continued to demonstrate capacity to work effectively with First Nations communities to identify and to assess EPH risks, and that such activities can result in recommendations for First Nations communities and other stakeholders with respect to addressing environmental risks. In addition, there has been continued progress over the past five years with First Nations communities developing their own capacity to identify and assess EPH risks. There has been less progress in communities undertaking of activities required to address EPH risks, which can be complex, and require considerable resources from other funding sources (i.e., outside of the mandate of the EPHP). While there are some areas where this third step is occurring (e.g., short-term remedial actions such as drinking water advisories and food safety precautions), overall there remain challenges for communities to address the more complex EPH risks (e.g., long-term actions involving housing and systems for drinking and wastewater).

The contingencies and factors involved in addressing EPH risks are numerous, and include the consideration of community prioritization and the availability of resources, all of which are external to the EPHP's mandate. This indicates that if the EPHP is expected to contribute to communities addressing their more complex EPH risks, then the program will need to continue coordinating and collaborating with the multitude of other stakeholders involved that assist First Nations communities in determining their priorities and obtaining resources to address EPH risks.

The evaluation was unable to directly assess the longer term outcome of improved health and well-being of First Nations. There is consensus among program representatives that the EPHP is likely contributing to improving health status by taking specific actions towards certain environmental public health risks.

Demonstration of Economy and Efficiency

The EPHP demonstrates some aspects of economy and efficiency in its delivery. Given the manner in which budgets are developed and resources allocated, it is challenging to determine the extent to which planning and budgets align with actual expenditures in each of the program's regions and activity areas. This also contributes to challenges with calculating some measures of efficiency using the financial data available for the program such as ratios of cost per output.

The program's strong, positive connections with First Nations communities and organizations, particularly at the community level, assist in ensuring the relevancy of the EPHP's services and advice. The EPHP is expending resources on EPH activities that are contributing to outcomes important to First Nations communities.

A key component of the EPHP successes can be attributed to the work of Environmental Health Officers (EHOs). While there were numerous areas of efficiency demonstrated for EHOs, the sustainability of their efforts may be impacted by an increasing workload in the area, and significantly greater demands being placed on the EPHP due to: an increased awareness of EPH risks in many communities; an increase in funding for community infrastructure projects; and, the ongoing growth and development of First Nations communities. The impact of workload pressures among EHOs is somewhat varied by region due to differences in capacity and priorities between First Nations communities.

Regarding matters pertaining to the program's performance measurement, the EPHP has generally improved during the past five years. Areas for improvement identified in the previous evaluation show that attempts have been made to advance reporting from strictly counting outputs to also including outcomes. In addition, there was work on the development of some of the data collection systems. Despite these improvements, there remain some challenges with respect to the comprehensiveness of data (e.g., different data collection tools and coverage parameters among regions) and the utility of data collected (e.g., focus on counting inspections but not systematically capturing the content of recommendations).

Recommendations

Recommendation 1

Explore options to enhance collaboration with stakeholders to support First Nations in addressing and mitigating complex environmental health risks (e.g., those that may require longer-term and more costly interventions). While there are many coordination and collaboration activities underway already, it is clear that for the intermediate (and thus longer term) outcomes to be achieved there needs to be continued involvement by EPHP in contributing evidence and assessments to support or promote the mitigation of complex environmental public health risks. In order to better assess the degree to which outcomes are achieved, the impacts of EPH collaboration and coordination on EPH risk mitigation should be documented, with the understanding that whether such risks are actually mitigated is often outside the EPHP's control.

There are numerous other stakeholders involved in various areas of EPH in First Nations communities. These stakeholders include: First Nations communities, various First Nations organizations, other federal government departments and agencies (e.g., INAC, EC, PHAC, CFIA, CMHC), various branches within Health Canada, and provincial governments. INAC and Health Canada require collaboration specifically in a number of areas associated with EPH given their combined contributions to achievement of EPHP objectives. For example, while INAC is the lead on the First Nations Water and Wastewater Action Plan, Health Canada has important supporting roles. Similarly, investigations by EHOs or reviews by public health engineers can result in recommendations for infrastructure improvements that are being funded through INAC programming. EPHP is only one of the players in this complex matrix; however, the EPHP is well-positioned to continue to provide evidence and information required to promote the mitigation of complex EPH risks, given their expertise and strong working relationships and connections to the community. EPHP's coordination and collaboration activities should continue at various levels including community, sub-region, region and national levels.

Recommendation 2

Review the current EHO allocation and workload within and across regions to determine sustainability of service to communities considering various forecasted additional demands and complexities. Building on the findings from the workload analyses currently underway, it will be important for the EPHP to clearly understand the challenges involved resulting from the current EHO allocation and workloads at both regional and national levels. This review would need to take into account the anticipated additional demands and complexities for many First Nation communities as a result of various changing contexts (e.g., increased infrastructure funding, climate change, economic development, population growth).

Recommendation 3

Produce an assessment of the impact of the recommendations made by EHOs to Chiefs and Councils in the area of food, water and housing. Given that the recommendations that EHOs develop are an essential output to achieving some of the desirable outcomes for the program, they are potentially a rich source of information and data for various uses at multiple levels. This would likely enhance the utility of the performance and monitoring data being collected by the program.

Management Response and Action Plan (or Management Response)

Evaluation of the First Nations and Inuit Health Branch's Environmental Public Health Program 2010-2011 to 2015-2016
Recommendations Response Action Plan Deliverables Expected Completion Date Accountability Resources

Recommendation as stated in the evaluation report

Identify whether program management agrees, agrees with conditions, or disagrees with the recommendation, and why

Identify what action(s) program management will take to address the recommendation

Identify key deliverables

Identify timeline for implementation of each deliverable

Identify Senior Management and Executive (DG and ADM level) accountable for the implementation of each deliverable

Describe the human and/or financial resources required to complete recommendation, including the source of resources (additional vs. existing budget)

Review the current EHO allocation and workload within and across regions to determine sustainability of service to communities, considering various forecasted additional demands and complexities.

Program management agrees with the recommendation. Efforts to improve the availability of, and access to, high quality data to promote better decision-making and planning is a priority of FNIHB. Within this context, FNIHB has developed an EHO Workload Assessment Tool to assess sustainability and determine future requirements for EHOs.

Validate EHO Workload Assessment Tool and prepare analysis of EHO workload.

Presentation to senior management on EHO Workload Analysis

September 2017

Tom Wong

Executive Director, Office of Population and Public Health

No additional resources required.

Produce an assessment of the impact of the recommendations made by EHOs to Chiefs and Councils in the area of food, water and housing.

Program management agrees with the recommendation.

A representative sample of recommendations in the areas of food, housing and drinking water will be reviewed to assess their impacts and determine if required follow up actions are taken.

If necessary, based on the review of the implementation of recommendations, policies and procedures will be revised in order to improve implementation of mitigation measures to address EPH risks.

Complete analysis and develop an approach to assess the impacts of a representative sample of recommendations in the areas of food, housing and water, to determine if required follow-up actions are taken, over a period of 6 months.

Update existing policies and procedures to improve implementation of recommendations by EHOs in order to address EPH risks.

Presentation to senior management on approach. March 2017

Tom Wong

Executive Director, Office of Population and Public Health

No additional resources required.

Initiate data collection in selected Regions or communities June 2017
Presentation to senior management on results February 2018
Updated policies and procedures April 2018

Explore options to enhance collaboration with stakeholders to support First Nations in addressing and mitigating complex environmental health risks (e.g., those that may require longer-term and more costly interventions).

Program management agrees with the recommendation.

EPHP (National Office and Regions) has a long history of collaboration with partners to work towards the mitigation of complex environmental public health risks (e.g., those requiring funding investments and/or longer-term interventions).

In the context of INAC's new infrastructure funding for water and wastewater, and their renewed mandate to address Drinking Water Advisories (DWA) on-reserve, a key EPHP priority is to collaborate with INAC to promote the prioritization of investments that address high risk systems from the health perspective. Public health advice and analysis in this area is already being shared at the working level and ADM level.

Establish and document systems and processes to share public health information and advice with INAC regarding strategic water infrastructure investments and drinking water advisories, at the National and Regional levels.

INAC-HC Joint Work plan on strategic collaborative activities related to new infrastructure investment planning and strategies to address DWAs.

March 2017

Tom Wong

Executive Director, Office of Population and Public Health

No additional resources required.

1.0 Evaluation Purpose

The purpose of this evaluation was to assess the relevance and performance (effectiveness, efficiency, and economy) of Health Canada's Environmental Public Health Program (EPHP) for the period of 2010–11 to 2014–15. The evaluation was conducted in accordance with the Treasury Board Secretariat of Canada's 2009 Policy on EvaluationEndnote ii and the Financial Administration Act. The evaluation is part of the Five-Year Evaluation Plan of Health Canada and the Public Health Agency of Canada.

An evaluation of the EPHP (known previously as the Environmental Health and Environmental Research Cluster) was first conducted in 2011, and it focussed on many of the same issues covering the fiscal period 2005-06 to 2009-10.Footnote 3 This evaluation presents an update on the progress of outcomes and changes made to the EPHP since the previous evaluation, and highlights program achievements, lessons learned, and delivery challenges.

2.0 Program Description

2.1 Program Context and Narrative

Conditions in the environment, both natural and human-built, can affect a person's ability to achieve and maintain good health. A healthy environment includes safe water and food supplies, properly designed, constructed and maintained housing and community facilities, as well as suitable treatment and disposal of wastewater and solid waste. To maintain a healthy environment, it is also necessary to plan for and respond to emergencies and work to prevent and control environmental communicable diseases.

Improving the health of Indigenous people is a shared responsibility among federal, provincial and territorial governments, as well as First Nations and Inuit communities and individuals. The EPHP must, therefore, continue to work collaboratively with a variety of organizations, such as:

  • First Nations communities and organizations at the local, regional and national levels;
  • Other areas of Health Canada and the First Nations and Inuit Health Branch;
  • Other federal departments, including: Indigenous and Northern Affairs Canada (INAC); Environment and Climate Change Canada (EC); Public Health Agency of Canada (PHAC); Canadian Food Inspection Agency (CFIA); and, the Canada Mortgage and Housing Corporation (CMHC); and,
  • Provincial and territorial governments.

The long term expected outcome for the EPHP is improved health and well-being of First Nations individuals and communities. A logic model depicting how the program intends to achieve this outcome was updated in 2015 (see Appendix 2). This evaluation assessed the degree to which the defined outputs and outcomes in the logic model were achieved over the evaluation timeframe. The updated EPHP logic model in 2015 is very similar to the one used in the 2011 evaluation of the EH-ER Cluster.

2.2 Program Profile

The primary objective of the EPHP is to "identify, address, and/or prevent human health risks to First Nations and Inuit communities associated with exposure to hazards within the natural and built environments." Within Health Canada, the EPHP is coordinated regionally by Environmental Public Health Services (EPHS) and supported nationally by the Environmental Public Health Division (EPHD). All program activities are provided in agreement with and by request of First Nations Authorities. In First Nations communities south of 60º (the main focus of this evaluation), EPH programming is delivered by Environmental Health Officers (EHOs) employed by Health Canada or First Nation communities and/or Tribal Councils in accordance with the National Framework for the Environmental Public Health Program in First Nations Communities South of 60º. All EHOs must have a Certificate in Public Health Inspection (Canada), which is recognized by health organizations in Canada as evidence of satisfactory training and competency. They may also participate in Continuing Professional Competencies in order to maintain their skills, knowledge and expertise.

Key programming under public health assessments continues to include activities that focus on eight core areas (Appendix 1): Drinking Water; Wastewater; Solid Waste Disposal; Food Safety; Housing; Facilities Inspections; Environmental Communicable Disease Control; and, Emergency Preparedness and Response. The role of the EPHP is to assist communities by providing training and education around EPH risks according to community priorities, developing recommendations for addressing EPH risk based on investigations, and reviewing infrastructure plans from a public health engineering perspective. Key activities include public health inspections and assessments, public education and training, and providing advice and guidance.

Environmental public health surveillance and risk analysis programming includes community-based and participatory research on trends and impacts of environmental factors, such as chemical contaminants and climate change on the determinants of health (e.g., biophysical, social, cultural, and spiritual).Footnote 4 This programming is directed to First Nations communities south of the 60th parallel, and in the case of the Northern Contaminants Program and the Climate Change and Health Adaptation Program, also to Inuit and First Nations north of the 60th parallel.Footnote 5

Between April 1, 2010 and March 31, 2015, the EPHP expended approximately $162 million through set contribution funding agreements and direct departmental spending to identify, assess and address human health risks associated with exposure to hazards within the natural and built environments. One key program difference as of 2014 is that Health Canada's EPHP activities are now performed in British Columbia by the First Nations Health Authority, as a result of the British Columbia Tripartite Framework Agreement on First Nation Health Governance. Another difference as of 2011 is that the former structure of two groups of programming (Environmental Health and Environmental Research) have now been integrated into one program, namely the Environmental Public Health Program at FNIHB. In addition, research and surveillance activities have focussed primarily on environmental contaminants and laboratory activities (see Appendix 2, EPHP Program Logic Model).

2.3 Program Alignment and Resources

The EPHP is listed as sub sub program number 3.1.2.2 under Health Canada's Program Alignment Architecture, and it contributes to the following strategic outcome: First Nations and Inuit communities and individuals receive health services and benefits that are responsive to their needs so as to improve their health status.

The Program's financial data for the years 2010-11 through 2014-15 is presented below (Table 2). Overall, the Program had expenditures of approximately $162 million over five years.

Table 1: Program ExpendituresTable 1 - Footnote a ($)Table 1 - Footnote *
Year Gs & CsTable 1 - Footnote b O&MTable 1 - Footnote c SalaryTable 1 - Footnote d TotalTable 1 - Footnote e
2010-2011 15,700,304 7,367,832 10,367,299 33,435,435
2011-2012 15,007,787 6,554,327 11,071,277 32,633,391
2012-2013 15,006,828 5,742,165 10,192,278 30,941,271
2013-2014 14,787,800 6,029,588 10,873,888 31,691,276
2014-2015 14,554,559 5,601,485 13,009,365 33,165,409
Total 75,057,278 31,295,397 55,514,107 161,866,782

3.0 Evaluation Description

3.1 Evaluation Scope, Approach and Design

The scope of the evaluation covered the period from April 1, 2010 to March 31, 2015 and included the environmental public health activities conducted by the Environmental Public Health Division (EPHD) and FNIHB Regions, including progress made on activities covered under the First Nations Water and Wastewater Action Plan (evaluation completed in November 2013).Footnote 6 Given that other evaluationsFootnote 7 have assessed the EPHP's activities specific to Inuit communities, this evaluation focussed only on First Nations communities. The evaluation excluded activities that are now covered under the British Columbia Tripartite Framework Agreement on First Nation Health Governance, which came into full effect in 2014-15. Also, the evaluation did not include FNIHB activities covered under the following horizontal initiatives:

  • Clean Air Agenda, Adaptation Theme (assessed concurrently as part of an Environment Canada horizontal evaluation),Footnote 8 and
  • Action Plan to Protect Human Health from Environmental Contaminants (evaluation completed in February 2014).Footnote 9

The evaluation aligns with the Treasury Board of Canada's Policy on Evaluation (2009) and considered the five core issues under the two themes of relevance and performance, as shown in Appendix 4. Corresponding to each of the core issues, specific questions were developed based on program considerations and these guided the evaluation process.

An outcome-based evaluation approach was used to assess progress made towards achievement of the expected outcomes, to identify any unintended consequences, and to document lessons learned. Given the EPHP was previously evaluated in 2011,Footnote 10 particular emphasis was placed throughout the present evaluation on determining what changes and progress had been made since the previous evaluation.

Data for the evaluation were collected using various methods, including: a review of literature, program documents, and administrative data; key informant interviews with both internal respondents (n=15), and respondents external to Health Canada (n=26);Footnote 11 a national survey of EHOs (n=69); and, a comparative analysis with other environmental public health services. More specific details on the data collection and analysis methods performed are contained in Appendix 4. In addition, data were analyzed by triangulating information gathered from the different methods listed above. The use of multiple lines of evidence and triangulation was intended to increase the reliability and validity of the evaluation findings and conclusions.

3.2 Limitations and Mitigation Strategies

All evaluations face constraints that may have implications for the validity and reliability of evaluation findings and conclusions. The following table outlines the limitations encountered during this evaluation. Also noted are the mitigation strategies put in place, so that the evaluation findings can be used with confidence to guide program planning and decision making.

Table 2: Limitations/Challenges and Mitigation Strategies
Limitation/Challenge Impact Mitigation Strategy
Baseline data on the health status of First Nations individuals living on-reserve is not linked directly to the activities or time period (2010-2015) covered by the evaluation. Evaluating the program's expected outcome to improve the health status of First Nations individuals and communities is challenging to accurately assess during this time period. This report uses proxy indicators, as well as data on the health status of First Nations individuals and communities for the period just prior to the evaluation.
While the participation of key informants from First Nations communities was sought, encouraged, and voluntary, the evaluation team was unable to interview the number of First Nations community representatives initially planned. The perspectives of First Nations community representatives may not be fully represented I n the evaluation's findings. This report notes when findings may not include the perspective of groups underrepresented in key informant findings.
Over the evaluation timeframe, the program's context changed following the introduction of new federal priorities. The evaluation took place during a change of government, where priorities shifted, such as: addressing the calls to action from the Truth and Reconciliation Commission of Canada; a review of the Safe Drinking Water for First Nations Act; and, increased resources to address health disparities for Indigenous people and other Canadians. Relevance of the program was assessed for two time periods: one period with the evaluation scope, and current priorities to assist in the development of forward-looking recommendations.
Detailed performance data for the EPHP were not available for all years covered by the evaluation. The availability of program performance data is limited. Gaps in documented performance data were filled with information from other lines of evidence (e.g., survey, interviews) where possible.

4.0 Findings

4.1 Relevance: Issue #1 – Continued Need for the Program

There is a continued need to identify and address human health risks to First Nations communities associated with exposure to hazards within the natural and built environments. This evaluation found that many of the current and projected environmental health risks facing First Nations communities are very similar to those that were outlined during the previous evaluation.

Similar to the previous evaluation, all lines of evidence (literature and document reviews, interviews and survey results) indicate that there remains a clear and consistent need to identify, assess, and address environmental health risks in First Nations communities. As identified in the previous evaluation, many First Nations communities experience significant EPH risks compared with other Canadian communities. The link between housing quality and health has been established and is well documented in the literature/document review. Indoor air quality, safety and overcrowding can have a direct impact on health. Over the last decades, First Nations communities have been faced with health problems as a result of the quality of their housing stock. For example, the First Nations Regional Health Survey (RHS), using data collected in 2008-2010, found that for many First Nations living on reserve, housing and living conditions are substandard with approximately one-quarter of First Nations adults living in over-crowded housing (23.4%), representing a substantial increase since the previous RHS in 2002 (17.2%). In comparison in the general Canadian population, 7% of adults live in over-crowded housing. High household membership or over-crowding is linked with various negative health outcomes such as mould exposure, and transmission of infectious disease.Footnote 12

A survey conducted in 2013 with First Nations respondents found that the majority of First Nations people on reserve feel that environmental issues affect their/their family's health "a great deal", and that this proportion has grown since 2010 (52 per cent, up from 42 per cent).Footnote 13 Among the eight core areas of EPH programming, the highest priority areas of concern have been directed to risks associated with housing and water infrastructure issues. As noted in the 2013 survey, the environmental issues that pose the greatest health concern to First Nations people on-reserve are water pollution, air pollution, and mould, each identified by between one in five and one in three. These were also the top three issues identified in 2010. More recently, the Standing Senate Committee on Aboriginal Peoples released a report on its study of on-reserve housing and infrastructure. Among its findings, the Committee heard testimony and visited First Nations communities with "boil-water advisories that had been in place for more than a decade, over-flowing sewage lagoons, roofs covered by tarps because there is no money to repair them, and small overcrowded bungalows where between 16 and 18 people sleep at one time"Footnote 14 which raises other possible EPH risks like the potential spread of communicable diseases, mould, and indoor air quality.

As illustrated in Table 3, findings from the current survey of EHOs were relatively similar to those from the previous evaluation's findings. The most frequently cited risk areas across both cohorts were housing, drinking water, and wastewater related risks, and none of the groups interviewed or surveyed identified large-scale changes or differences in areas of risk that had occurred over the past five years.

Table 3: Most Frequently Cited EPH Risk Areas by Environmental Public Health Officers
EPH Area 2011 (n=93) 2016 (n=65)
Housing conditions 92% 99%
Drinking water 91% 95%
Wastewater 80% 72%
Food safety 53% 66%
Solid waste 47% 52%
Communicable disease control 43% 39%
Community emergencies 20% 29%

Note: Facilities inspection, while a core program area, addresses various risk areas.

Source: EHO Survey 2011; EHO Survey, 2016

As illustrated in Table 4, among those identifying the above risks in 2016, drinking water and housing risks were most frequently rated as "high priority".

Table 4: Most Frequently Cited as "High Priority" Risk Areas
Core EPH issues identified as Risk Areas Overall % of Survey Respondents Identifying Area as a  High Priority
(n=65)
Drinking Water Risks 88%
Housing Risks 80%
Wastewater risks 38%
Food safety risks 35%
Communicable disease control risks 25%
Solid waste risks 14%
Community emergency risks 12%
Note: Multiple responses allowed, therefore, will not add to 100%

With respect to drinking water and wastewater, responses from the EHO survey identified that challenges continued to result from poor or old infrastructure; improper infrastructure maintenance; lack of training and availability of certified operators; systems working at or beyond capacity; climate change; local geography characteristics; and numerous socio-economic issues.

4.2 Relevance: Issue #2 – Alignment with Government Priorities

Similar to the previous evaluation, the EPHP activities continue to align with federal priorities to address environmental public health risks in First Nations communities. The EPHP is aligned with Health Canada's Strategic Outcome 3: First Nations and Inuit communities and individuals receive health services and benefits that are responsive to their needs so as to improve their health status.

The previous evaluation found that the EH-ER Cluster activities aligned with the federal and Health Canada priorities related to environmental public health. Since April 2010, various Government of Canada commitments have supported improving the identification and mitigation of environmental health risks, particularly in the areas of drinking water and wastewater. For example, the 2011 and 2013 Speeches from the Throne outlined the Government's continued work with Indigenous peoples. Budget 2011 outlined funds to support water, housing and First Nations and Inuit health with Budget 2012 through 2015 mentioning support for various aspects of First Nations and Inuit health. More recently, Budget 2016 has included additional commitments specifically in the areas of drinking water, wastewater, solid waste management and housing in First Nations communities.

4.3 Relevance: Issue #3 – Alignment with Federal Roles and Responsibilities

The EPHP role is advisory in nature and has not changed significantly since the previous evaluation. However, there may be potential in the future to move towards a more regulatory role in the area of drinking water.

Health Canada (FNIHB) provides advice, recommendations and guidance to First Nations communities on how to reduce or mitigate potential environmental public health risks in the natural and built environments. In 2012, the Federal Government introduced Bill S-8, the Safe Drinking Water for First Nations Act, with the Act receiving Royal Assent on June 19, 2013 and coming into force on November 1, 2013. The Act enables the Federal Government to develop enforceable federal regulations for access to safe, clean, and reliable drinking water; the effective treatment of wastewater; and the protection of sources of drinking water on First Nations lands. At the time of the finalization of this evaluation report, the Act was under review by INAC Ministerial direction, and the future of the regulatory development project was unknown.

Currently, in the absence of a statutory regime or legislative framework, the activities delivered under the EPHP remain advisory. Health Canada also does not currently have the mandate to provide funding to communities to address or remediate risks, as this is the purview of INAC.

With respect to the other core areas of EPH mitigation, the findings from the current evaluation are very similar to those from the previous evaluation. Health Canada plays an advisory role, and the delivery of this role varies according to region. As a result, activities under the EPHP are provided at the request and/or with the agreement of First Nations authorities. EPHP personnel operate with the understanding that Health Canada does not have any EPH authorities to enforce existing public health standards or regulations (federal, provincial, otherwise) on-reserve.

As outlined in the previous evaluation, there are numerous other stakeholders involved in various areas of EPH in First Nations communities. These stakeholders have not changed substantially over time and include: First Nations communities, various First Nations organizations, other federal government departments and agencies (e.g., INAC, EC, PHAC, CFIA, CMHC), various branches within Health Canada, and provincial governments. INAC and Health Canada require collaboration specifically in a number of areas associated with EPH given their combined contributions to achievement of EPHP objectives. For example, while INAC is the lead on the First Nations Water and Wastewater Action Plan, Health Canada has important supporting roles. Similarly, investigations by EHOs or reviews by public health engineers can result in recommendations for infrastructure improvements that are being funded through INAC programming. For the management of drinking water quality, First Nations are responsible for the construction, operation and management of the systems; INAC sets the standards for these systems and provides funding for the construction, operation and maintenance of the systems; and HC provides support to protect public health, including drinking water quality monitoring and funding of Community-Based Water Monitors. For example, if the HC's Environmental Health Officer's(EHO) review and interpretation of drinking water quality results indicate that drinking water is not safe, the EHO immediately communicates recommendation(s) (such as a "boil water" advisory) to the Chief and Council for their action.

The provision and management of housing on reserve lands is the responsibility of First Nations, with support from the Government of Canada. In addition to government funding from INAC, First Nations are encouraged to identify funding from other sources for their housing needs, including shelter charges and loans. According to documentation, and confirmed through key informant interviews and the EHO survey, despite the large number of stakeholders involved, roles and responsibilities are clearly outlined through various agreements and arrangements depending on the stakeholders involved, with limited overlap or duplication.Endnote iii

Echoing the findings from the previous evaluation, the one area in need of greater attention and highlighted by many of the key informants from First Nations communities and organizations is the continued gap between known risks that have been identified and assessed by Health Canada, and then those risks actually being addressed by the contributions of others. For example, the link between Health Canada's identification and assessment work and the contribution of others like INAC's programming and funding that would be needed to address the more complex risks identified and assessed by the EPHP (e.g., housing and infrastructure changes). This challenging gap was also identified in many of the interviews with Health Canada representatives, and was outlined in open comments by some respondents to the EHO Survey.

4.4 Performance: Issue #4 – Achievement of Expected Outcomes (Effectiveness)

The immediate outcome of "improved environmental public health risk identification" was defined according to four sub-outcomes of:

  • Timely data collection, analysis, dissemination and reporting;
  • Evidence based decision-making;
  • First Nations awareness of environmental public health risks; and
  • Immediate risk mitigation activities.

Timely data collection, analysis, dissemination and reporting

There are multiple layers of data collection, analysis, dissemination and reporting within the EPHP. At the community level, and similar to the previous evaluation, this process continues to be working well, as EHO expertise and knowledge connects with community priorities to guide the collection, analysis, dissemination and reporting undertaken within a specific community. Data collected at the community level is analyzed and reported through the form of recommendations to Chief and Council. Findings are disseminated through various training and community awareness activities. At the program level, there has been some improvement in data collection systems since the previous evaluation, and continued work in the analysis, dissemination and reporting of the program results, particularly with respect to drinking water. The current evaluation found gaps and challenges that relate to needs for greater data comprehensiveness at a national level, and the utility of data in being able to report on outcomes and use in risk management.

Data collection, analysis, dissemination and reporting occur at various levels under the EPHP program. EHOs work directly with communities through investigations and various other monitoring activities to collect data on aspects of a specific community (e.g., facility, housing, water system) that they then analyze and report back to the community through recommendations to Chief and Council. Further dissemination of results is continued through work with community health representatives, and potentially through related training and awareness activities for community representatives.

Another level of data collection, analysis, dissemination and reporting occurs at the program level, with some aspects of the EHO and CBWM activities getting captured within regional and national data systems. Analyses of these data then feed into various regional and national performance reports (e.g., FNWWAP national reports; EPHP Performance Report).

Data systems

The key data systems in use by the Program to identify and assess risk are the WaterTrax and Environmental Health Information System (EHIS). For various reasons, some regions (e.g., Alberta, Quebec) use alternative data systems, but much of the data from these systems can be rolled up at a national level. The previous evaluation found that there were challenges with the data systems in place at that time, but that considerable progress had been made over the five-year period. EHIS was implemented in 2005, and drinking water data were being compiled in various regional data systems.

For the current evaluation, overall, many Health Canada key informants perceived that the timeliness and quality of data systems for the EPHP had improved during the period under study. Informants indicated that improvements were largely the result of increased attention, training and diligence of regions and EHOs in populating the data systems. According to the FNWWAP Drinking Water Performance Indicators National Report, in 2012-13 all regions had access to a drinking water database. Maintenance of regional databases is necessary, so that drinking water quality results can be communicated in a timely manner, allowing for informed and timely decisions such as recommending the issuance of a drinking water advisory (DWA).

When Health Canada key informants identified weaknesses with the data / data systems, these generally related to the comprehensiveness of the data for national reporting purposes, and the utility of the data for decision-making.

Key informants' concerns about data comprehensiveness included the fact that EPHP does not have access to data from First Nations communities that have entered into a Health Funding Agreement, and therefore cannot report on or conduct analysis of risks facing all First Nations communities at the regional or national levels. Additionally, existing systems are not currently designed to allow for the capture of all relevant EPH activities or risks related to any given community, facility, system or home. As a result, risk analyses and resulting decision-making may at times be based on partial information. Finally, there are multiple systems in place and these do not communicate with each other. EPH data is currently housed in several different databases depending on the program activity and the region, making it difficult to roll-up and analyze regional and national data, and identify risks and trends. For example, the drinking water monitoring data and drinking water advisory data are housed in different databases depending on the Region. Quebec uses Water-Eau, Alberta uses ELPHIS, and the four other Regions use WaterTrax. For data related to public health inspections, investigations, and other activities undertaken, Alberta uses Hedgehog, while the rest of the regions use EHIS. There is no communication or interoperability between systems; data is not standardized and must therefore be rolled up manually for national reporting; and there is little capacity for integrated, complex or comprehensive analyses of environmental health risks and trends across systems/program areas/activities, which would be useful at all levels of decision-making, from First Nations to senior management.

Key informant concerns about data utility tended to focus on the perceived limitations of the data systems in capturing outcomes, and in providing strategic information useful for risk management and decision-making. The data are not yet being used optimally to monitor environmental public health risks and outcomes, or to support risk management. At the community and regional levels, data pertaining to drinking water quality monitoring results is housed separately from data pertaining to all other EPH program areas (inspections, investigations, assessments, surveillance related to facilities and housing). These databases do not speak to each other and do not incorporate geographic information systems data in a way that would facilitate cross-referencing or integrated analysis of risk trends and results across the different program areas. Further, these systems do not integrate data from other systems/departments (e.g., those used in other FNIHB Regions, or those used by INAC in relation to housing or water and wastewater infrastructure projects and reporting. Some additional specific issues mentioned by a few respondents included the inability of the inspections data systems to facilitate risk assessment or prioritization (e.g., provide risk rankings and follow-up schedules on the basis of inspection results).

Staff

Having an adequate number of EHOs is considered to be a possible contributing factor to more timely data collection, analysis, dissemination and reporting at both the community and program level, which should lead to improved environmental public health risk identification. The previous evaluation identified a challenge with the demands for EHOs and their workload, to the extent that there were issues with having to work reactively rather than proactively in some communities. While the current evaluation identified continued challenges related to EHO workload, it is unclear whether the actual number of EHOs has increased from the previous evaluation.

Inspections, Monitoring and Investigations

Water sampling activity is an indicator of data collection, analysis, dissemination and reporting which should lead to improved environmental public health risk identification. While the document review did not obtain data for all years covered by the evaluation, from the information received it appears that the amount of water sampling is increasing overall in First Nations communities, with compliance rates for recommended frequency of sampling increasing as well. According to the 2015 evaluation of the CBWM Program, "there has been a 22% total increase in the number of bacteriological samples collected for all distribution systems from FY2004-05 (76,029 samples) to FY2011-12 (92,785 samples)." Footnote 15 The same evaluation found that while the recommended sampling frequency set out in the Guidelines for Canadian Drinking Water Quality (GCDWQ) was generally not met at a national level over the period from 2008-09 to 2011-12, compliance increased annually from 42% in 2008-09; 45% in 2009-10; 58% in 2010-11; to 60% in 2011-12. However, in 2014-15, the compliance rate declined to 42%. This same trend was found in the available National FNWWAP Reports with the number of water samples analyzed increasing by approximately 6% overall (from 134,659 in 2011-12 to 142,853 in 2012-13). Although compliance rates increased over the 2008-09 to 2011-12 period, there was a sharp decline in 2014-15, highlighting that further improvement is required to meet the recommended monitoring frequencies for bacteriological parameters of the Guidelines for Canadian Drinking Water Quality. HC's EHOs work directly with First Nations communities to increase monitoring compliance for bacteriological parameters in distribution systems with 5 or more connections by identifying targeted communities; addressing non-compliance (eg.,identify suitable back-up CBWMs) and; in the absence of a CBWM, identifying/training a replacement while assisting in monitoring to the extent possible.

The delivery of public health inspections is a core component of EPH services in First Nations communities. Overall, there are similar proportions of routine inspections and requested inspections; however, this proportion differs between communities and across facility types and can vary considerably from year to year, and from region to region.Footnote 16 According to the 2014-15 Performance Indicators Report, there were on average approximately 7,000 inspections each year for the period covered by the evaluation. The variability is demonstrated with a low of 5,960 inspections in 2012-13 and a high of 9,175 inspections in 2011-12. The number of inspections is slightly lower, but comparable to the previous evaluation, which identified the number of annual inspections ranged from 8,089 (2009-2010) to 8,877 (2007-08).

While the proportion changes slightly from year to year, the number of requested inspections is usually slightly higher than routine inspections. For example, in 2013-14, the proportion of inspections that were requested was 54% compared with the 46% that were categorized as routine inspections. The majority of requested inspections are in the areas of housing, sewage, and drinking water. In contrast, the routine inspections generally occur for food facilities and community care facilities (childcare, schools and other care facilities). According to a recent report analysing EHO workloads, in 2013-14, the need for some regions to focus on emergency activities (e.g., flooding) resulted in EHOs only being able to meet approximately one-third of their targeted inspection regime (i.e., routine inspections as mandated through National Framework).

EHOs work with First Nations Authorities and other public health agencies to address suspected or confirmed cases or outbreaks of foodborne, waterborne and vectorborne diseases. According to EPHP Performance Reports, the vast majority of the activities related to communicable disease investigations are related to investigating vectorborne illness resulting from rabies risks and animal bites from dogs. The number of these investigations appears to have more than doubled from 135 in 2010-11 to 301 in 2014-15. Activities around Lyme Disease investigation and awareness increased in the last four years while the West Nile Virus work continued, resulting in a large increase in the total number of investigations for vectorborne diseases.

Zoonotic surveillance activities are primarily located in three regions (Atlantic, Ontario and Manitoba) and focus on mosquito, bird and tick surveillance. Surveillance activities seem to have decreased more recently from 378 activities in 2013-14 to only 86 activities in 2014-15Footnote 17. Overall, the workload of EHOs in these areas has increased. Footnote 18 An initial assessment of the gap in the current number of EHOs reveals a shortfall, thus leaving no surge capacity in most communities to address unexpected emergencies or events.

Evidence-based decision-making

The previous evaluation found that the EH-ER cluster was contributing to the evidence and research used to make decisions on EPH-related activities. Similarly, the current evaluation found that the EPHP activities undertaken were largely evidence-based. There was evidence of best practices being applied in the work with First Nations communities, research being conducted and disseminated, and an ongoing effort to update various guidance documents.

Overall, key informants across all respondent groups were of the view that EPH program activities were evidence-based, and that this had largely remained stable over time. Health Canada representatives most often pointed to the fact that program activities were aligned with evidence-based national guidelines (e.g., GCDWQ) and provincial EPH standards and practices (e.g., drinking water quality standards, themselves largely based on the GCDWQ). As well, some Health Canada respondents noted that EHOs and other areas of the program such as the research components rely on their internal networks, and also have access to Branch and external experts with expertise. A few Health Canada key informants noted that all EHOs are certified and part of a professional association that expects they are up to-date on current evidence-based practices.

Key informants from First Nations communities and organizations indicated no concerns with the evidence base of the Program's risk assessment and mitigation (e.g., the EHOs are held in high esteem in communities, reports are viewed as careful, rigorous and trustworthy). According to one respondent, First Nations communities hiring their own EHO through a Health Funding agreement are well-supported by the Health Canada regional office with regular communication on standards and case reviews.

According to the EHO survey, the majority of EHOs (55%) reported that their use of best practices from other jurisdictions, organizations and departments had increased over the past five years. Overall, 38% indicated that this has increased "somewhat" while an additional 17% of EHOs indicated that it had increased "significantly". A large proportion reported their use of best practices had remained the same (40%), while approximately 5% reported this having decreased.

In addition, the program generates evidence through a number of research initiatives. The main research initiatives that produced findings that were further disseminated to various communities, researchers, and internal HC audiences included:

  • First Nations Biomonitoring Initiative (ongoing),
  • Northern Contaminants Program (ongoing),
  • Drinking Water Quality Program (2 years),
  • First Nations Environmental Health Innovation Network (3 years),
  • First Nations Food, Nutrition and Environment Study (FNFNES) (ongoing), and
  • First Nations Environmental Contaminants Program (FNECP) (ongoing).

According to Health Canada key informants, data from these studies provide communities with an evidence base to understand risks of exposure to environmental contaminants within their community. According to Health Canada representatives, the research initiatives are largely community-driven (e.g., the FNECP) or will engage communities in methodology design and knowledge transfer.

From the interviews with First Nations communities, the small number of community representatives who had participated in the FNECP were pleased with the Program and confirmed that the program had helped to understand the nature of contaminants in their community and mitigation strategies (e.g., levels of mercury in local fish, uranium in ground and surface water). Many of the EPHP representatives identified improvements in the community-based research program, including guidelines that have been implemented in the call for proposals for research projects. Similarly, they noted that both national and community-driven research initiatives are customizable to the concerns of the community.

Interviews with First Nations organizations noted a few areas for improvement, such as ensuring that the notification of the AFN of planned research with First Nations communities occurs prior to implementation. For all major research work, FNIHB respected the AFN-FNIHB Consultation Protocol (including FNFNES, Biomonitoring and FNECP). However, there were times in the past where AFN was informed of one-off research projects after the fact, yet this was not a common practice.

The document review found various examples of policies, guidelines, frameworks and procedures that have been developed, updated or adapted by the EPHP during the period covered by the evaluation. Examples included:

  • Drinking Water Program Manual;Footnote 19
  • Housing Inspection Resource Manual for EHOs;Footnote 20
  • Field Reference Manual: Non-Microbiological Drinking Water Quality Parameters;Footnote 21
  • Guidance on ensuring the safe delivery of drinking water to individual homes and facilities where piped water from a central system is not available;Footnote 22 and,
  • A set of detailed and complete set of instructions in straightforward language for the installation, use, maintenance and decommissioning of cisterns for holding drinking water from a certified safe source.Footnote 23

The target audiences for these products vary including FNIHB regions, EHOs, CBWMs, First Nations communities and other stakeholders.

First Nations awareness of environmental public health risks

Over the past five years there appears to be an increased awareness of environmental public health risks among First Nations community members, along with increased awareness of EPH services in their community. This greater awareness has led to increased access and use of the EPH services in communities. Given the workloads and travel restrictions in some regions, this increased access and use may differ according to area and region. Awareness of the EPHP was lower among the regional First Nations organizations.

The previous evaluation demonstrated that considerable effort and emphasis was placed on the development of public education and awareness materials under the EH-ER cluster activities. The current evaluation found similar levels of effort for the subsequent five-year period.

The evaluation found numerous examples of public education and awareness materials that had been developed by the EPHP during the past five years. From the document review, it was noted that the materials tended to range in topics (drinking water, traditional food safety, infection control, mould) and were delivered in various formats (fact sheets, videos, websites, newsletters, posters, presentations, media releases) to various groups of community members (parents, youth, children, council members, facilities employees). According to the 2014-15 Performance Indicators Report, there were a total of 257 public education materials developed nationally in areas of drinking water, wastewater, housing food safety, and environmental communicable disease control.

A survey conducted in 2013 with First Nations respondents found that more than half of First Nations people living on reserves (54%) say, unprompted, that they recall having seen, heard, or read something about the impacts of the environment (outdoors and indoors) on their health in the previous 12 months. When asked, unprompted, about the topic of this information, the largest proportions of First Nations people mention drinking water safety (20 per cent), mould (18 per cent), and oil and gas pollution (16 per cent). Alternatively, a decrease in the number of First Nations on reserve unable to identify any environmental issue was also observed (16% of respondents during the 2013 survey compared to 31% in 2010).Footnote 24

From the interviews, Health Canada regional representatives and First Nations communities representatives named a variety of health promotion and public education strategies that were used in the communities to raise awareness: radio, door hangers, pamphlets/publications, public health meetings, booths at health fairs and community events, door-to-door interactions, training (food safety) and annual reports. Some community representatives noted that in-person strategies are most effective in terms of outreach and awareness-raising, and tapping into community gatherings is another common practice that is perceived to be effective and culturally acceptable/well-received. A few First Nations community representatives also indicated that awareness of EPH risks in some communities is increasing in part due to knowledge of external factors, such as impacts of oil and gas activity on drinking water quality.

The evaluation captured the perspective of EHOs who have the opportunity to observe possible changes in levels of awareness among First Nations individual community members. As illustrated in Figure 1, approximately two-thirds of respondents to the EHO survey (64%) reported that there had been a positive shift in the community members' awareness and knowledge of EPH risks in the past five years. Most EHOs (83%) reported that the EPHP had made at least some contribution to community members' levels of awareness of EPH risks. This rate is similar to that found in the previous evaluation where in 2011, 54% of EHOs noted a positive shift, and 87% indicated that the Program had made a contribution towards this shift.

Figure 1: Community members' knowledge/awareness of EPH risks and EPHP contribution

Figure 1A: Knowledge/Awareness of EPH Risks
Figure 1A
Figure 1A - Text Description

Figure 1A illustrates community members' knowledge/awareness of EPH risks and EPHP contribution (n=63). Approximately two third of survey respondents to the EHO survey (64%) reported that there had been a positive shift in the community members' awareness and knowledge of EPH risks., while 33% indicated that their awareness and knowledge of EPH risks stayed the same.

Figure 1B: EPHP Contribution
Figure 1B
Figure 1B - Text Description

This figure shows the EPHP contribution to community members' levels of awareness of EPH risks. Among the respondents, 22% stated that the EPHP has significant contribution, 60% stated that EPHP has some contribution, and 17% stated that it has little contribution.

Source: EHO Survey, 2016, (n=63).

Many key informants from across the respondent groups tended to agree that awareness of EPH risks has increased over the last five years among First Nations community members through the active dissemination of information, education and site visits. Most Health Canada regional representatives noted awareness has increased in part due to the information that is collected by the EHOs (through inspections, water sampling, and investigations) and reported back to the community leadership, along with risk mitigation strategies. Many Health Canada representatives noted that community awareness is also increased by means of the EHO working collaboratively with different groups, including Health Directors, Housing Director, nursing staff and a variety of community members, and not just Chief and Council.

Most First Nations community representatives interviewed indicated that their communities had good access to the program.This was confirmed with the findings from the EHO survey (Figure 2) with the majority of EHOs (57%) reporting that availability of EPH services for communities had improved during the previous five years. This is similar to the evaluation in 2011 in which 64% of EHOs indicated that availability of services had increased.

Most community representatives reported being satisfied with access to the EPHP services through their EHO, and indicated that the communications and responsiveness provided by EHOs are very good. In some cases, the EHO was perceived as taking a supporting role in the community (e.g., providing evidence to support applications for infrastructure funding to INAC). Their relationship-building efforts and site visits were viewed by many community representatives to be very important to the visibility and credibility of the program and their position with the community. Many community representatives indicated that through their role in capacity building and training, EHOs are perceived as valuable resources, professionals and trainers. It was noted in the EHO survey that the majority of respondents (53%) reported that communities access to skills development and training related to EPH had increased over the past five years (Figure 2). This is a similar rate to the 47% of EHOs reporting this in 2011.

Consistent with the previous evaluation, in general, the EPHP and in particular the EHOs are quite visible with many communities as reported by HC regions and confirmed by many communities. The interviews with First Nations regional organizations suggest that there is less awareness of the EPHP and the role of the EHO among some of the regional organizations interviewed. Interviews with some First Nations communities and organizations noted the need for greater promotion of the program within communities and regions, and a greater sharing of community experiences in addressing EPH risks. Venues indicated for this included regular meetings of EPHP representatives with Health Directors in specific areas or at a regional level.

Figure 2: Community awareness, availability and use of EPH services

Figure 2A: Community awareness of EPH services (n=63)
Figure 2A
Figure 2A - Text Description

This figure shows the community awareness of EPH services. There were 63 survey respondents. Among them, 33% stated that the community awareness increased significantly, 35% stated that the community awareness somewhat increased, 27% stated that the community awareness stayed the same, 3% stated that the community awareness somewhat decreased and 2% stated that it decreased significantly.

Figure 2B: Availability EPH services (n=63)
Figure 2B
Figure 2B - Text Description

Figure 2B illustrates the availability of EPH services. According to survey respondents who were aware of the availability of EPH services (n=63), 59% believed that EPH services has increased, with 37% stating that the services increased significantly, and 22% stating that the services increased somewhat.

Of note is that 22% stated that the services remained the same. Furthermore, 19% of survey respondents indicated that the availability of EPH services decreased. Whereby 13% stated that services decreased somewhat, 6% indicated that the EPH services decreased significantly.

Figure 2C: Use of EPH services by communities(n=63)
Figure 2C
Figure 2C - Text Description

Figure 2C illustrates the use of EPH services by communities (n=63). 65% of survey respondents indicated that EPH service utilization increased, with 40% stating the utilization increased somewhat and 25% indicating uptake increased significantly.

29% of survey respondents stated that their utilization stayed the same, whereas 6% indicated that the usage of EPH services decreased somewhat.

Figure 2D: Access to EPH skills dev't and training (n=55)
Figure 2D
Figure 2D - Text Description

Figure 2D illustrates the level of access to EPH skills and development training. 53% of survey respondents indicated that access to EPH skills development and training has increased, with 47% stating that access increased somewhat, and 6% indicating that access increased significantly.

Of note is that 29% of survey respondents stated that the access to EPH skills and development training remained the same. Moreover, 15% indicated that access decreased somewhat, while 4% stated that access decreased significantly.

Source: EHO Survey, 2016

Immediate risk mitigation activities

Similar to the previous evaluation, many EPH risks are being mitigated in the immediate or short term as a direct result of EHO's risk assessment activities, advice or recommendations. This appears to be happening more frequently now during drinking water monitoring activities, food handler training sessions, and community education sessions than during the previous evaluation time period.

Both in 2011 and 2016, EHOs reported frequently noticing EPH risks that are being mitigated or eliminated immediately due to their ongoing involvement with community members (Table 5). This is happening most often as a result of drinking water monitoring activities and food handler training sessions, but also occurring as a result of public health inspections, communicable disease investigations, and community education sessions. The extent to which this was reported by EHOs to have occurred was generally more frequently in 2016 during drinking water monitoring activities, food handler training sessions and community education sessions when compared with 2011 rates.

Table 5. Activity areas where EPH risk is mitigated or eliminated in the short term
Activity Frequently and Very Frequently
2011 (n=93)
Frequently and Very Frequently
2016 (n=49 to 65)
Drinking water monitoring activities (n=65) 66% 78%
Food handler training session (n=61) 59% 75%
Public Health Inspections (n=64) 36% 36%
Communicable disease investigation (n=49) 47% 45%
Facilities plan review (n=52) 54% 52%
Community education session (n=59) 29% 46%
Emergency planning or response activities (n=50) 25% 30%
Source: 2011 EHO Survey; 2016 EHO Survey – Activity column includes sample population from 2016 in brackets

A survey conducted in 2013 with First Nations respondents found that in response to a specific awareness campaign conducted by EPHP, more than one-third of First Nations (36%) indicated that they did something based on the campaign materials they saw. The largest proportions of First Nations took action to protect against harmful chemicals (19%) and to prevent mould in the home (19%), and talked to others about environmental health risks (17%).Footnote 25

Achievement of Intermediate Outcomes

Similar to the previous evaluation, there continues to be progress in increasing First Nations communities capacity to identify and assess EPH risks and to address certain risks through short-term remedial actions when addressing areas such as drinking water advisories and food safety issues. Consistent with the previous evaluation, progress is slower with respect to addressing more complex recommendations. Considerable effort is being made in the area of training and education around EPH risks.

The previous evaluation found that while First Nations community capacity to identify and assess EPH risks has increased, the community capacity to mitigate or address more complex risks had not increased to the same extent. The findings from the current evaluation are consistent with this previous finding. As illustrated in Figure 3, EHOs indicated that while there had been some change in capacity among communities to independently identify and assess EPH risks over the past five years, there was more limited evidence of change in communities capacity to either address and mitigate EPH risks, or manage and administer EPH programming under the guidance of EHOs. This pattern is very consistent with that found with the 2011 survey of EHOs.

Figure 3: Community capacity to identify, assess, address and mitigate EPH risks

Figure 3A: Community capacity to independently identify and assess EPH risks (n=57)
Figure 3A
Figure 3A - Text Description

Figure 3A illustrates the community (n=57) capacity to independently identify and assess EPH risks. 49% of survey respondents stated that their capacity to independently identify and assess EPH risks increased somewhat, while 39% indicated that their capacity stayed the same.

Overall, it appears that the community capacity to independently identify and assess EPH risks has increased by 51%, with only 9% stating that their capacity decreased somewhat.

Figure 3B: Community capacity to address and mitigate EPH risks (n=61)
Figure 3B
Figure 3B - Text Description

This figure shows the community capacity to address and mitigate EPH risks. There were 61 survey respondents. Among them, 34% stated that the capacity somewhat increased, 52% stated that the capacity stayed the same, 10% stated that the capacity somewhat decreased, and 2% stated that it decreased significantly.

Figure 3C: Community capacity to manage and administer EPH programming under the guidance of EHOs (n=57)
Figure 3C
Figure 3C - Text Description

This figure shows the community capacity to manage and administer EPH programming under the guidance of EHOs. There were 57 survey respondents. Among them, 2% stated that the capacity increased significantly, 35% stated that the capacity somewhat increased, 54% stated that the capacity stayed the same, 7% stated that the capacity somewhat decreased, and 2% stated that it decreased significantly.

Source: EHO Survey, 2016

According to some Health Canada regional representatives and some representatives from First Nations communities and organizations, short-term remedial actions and drinking water advisories lead consistently to timely reactions by communities, so that immediate behaviour changes can be made by community members. Similarly, issues with food safety are quickly taken into account. A few community representatives reported that they have had no food-related outbreaks in many years, suggesting that the food safety training is effective and changing behaviour.

Similar to what was highlighted in the previous evaluation, many of the representatives from First Nations communities and organizations and Health Canada regional representatives noted that recommendations that are more complex and require costly interventions are more difficult for the communities to respond to as these may require additional funding from INAC or investments by the community that they cannot afford due to other priorities. Some First Nations communities indicated that the human resources available to address and manage all EPH risks within their communities are inadequate, and that they would benefit from having trained resources on site to conduct monitoring and follow-up in various risk areas.

It was noted by some Health Canada representatives that an area for improvement with respect to the program is to enhance the risk-based approach to programming. Routine inspection and monitoring frequencies are important, but in some cases may not be required at current levels for certain facilities or systems that are extremely low risk. Conversely, high risk facilities and systems might require increased inspection or monitoring frequencies and it is important to help communities prioritize recommendations pertaining to such systems and facilities. As previously discussed, data systems might assist in this respect by facilitating risk assessment or prioritization (e.g., provide risk rankings and follow-up schedules on the basis of inspection results). An additional area for consideration outlined in interviews with EPHP representatives was to begin to track both the content and outcome of recommendations (e.g., monitor which recommendations were implemented, and which still require attention or follow-up).

Achievement of Longer-Term Outcomes

The evaluation was unable to directly assess the longer term outcome of improved health and well-being of First Nations. There is consensus among program representatives that the EPHP is likely contributing to improving health status by taking specific actions towards certain environmental public health risks; however, data is limited in this regard.

Most key informants across all respondent groups indicated that in their opinion the EPHP contributes to the improved health and well-being of First Nations' individuals and communities. The evidence offered by key informants to support this view often centered on the Program's activities, and to the extent that risks are being mitigated, immediate outcomes logically linking to the improved health and well-being outcome. Examples of activities cited included sampling of drinking water to verify quality and, as required, recommendation for drinking water advisories; and avoiding outbreaks of food-borne illness through food safety training and regular inspection of food facilities.

Following the stated program logic, improved health and well-being likely is occurring, particularly where intermediate outcomes are being achieved; however, the evaluation was not able to accurately ascertain this link due to limited data on First Nations health outcomes during this period.

4.5 Performance: Issue #5 – Demonstration of Economy and Efficiency

The Treasury Board of Canada's Policy on Evaluation (2009) and guidance document, Assessing Program Resource Utilization When Evaluating Federal Programs (2013), defines the demonstration of economy and efficiency as an assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes. This assessment is based on the assumption that departments have standardized performance measurement systems and that financial systems link information about program costs to specific inputs, activities, outputs and expected results. The financial information available consists primarily of the expenditures by the EPHP as outlined in Table 6.

Table 6: EPHP ExpendituresTable 6 - Footnote a by Region and HQTable 6 - Footnote *
2010-2011 to 2014-2015 ($)
Region Gs & CsTable 6 - Footnote b O&MTable 6 - Footnote c SalaryTable 6 - Footnote d TotalTable 6 - Footnote e %
Atlantic 3,259,065 2,169,913 4,330,263 9,759,241 6%
33% 22% 44% 100% -
Quebec 4,148,970 1,618,192 4,505,928 10,273,090 6%
40% 16% 44% 100% -
Ontario 8,517,078 10,100,411 11,216,452 29,833,941 18%
29% 34% 38% 100% -
Manitoba 9,438,823 3,835,898 5,573,052 18,847,773 12%
50% 20% 30% 100% -
Saskatchewan 21,505,886 2,400,488 3,798,492 27,704,866 17%
78% 9% 14% 100% -
Alberta 7,122,564 6,021,272 10,060,002 23,203,839 14%
31% 26% 43% 100% -
Northern 5,854,801 55,691 209,741 6,120,234 4%
96% 1% 3% 100% -
HeadquartersTable 6 - Footnote f 15,210,091 5,093,531 15,820,177 36,123,799 22%
42% 14% 44% 100% -
Total 75,057,278 31,295,397 55,514,107 161,866,782 100%
% 46% 19% 34%

Departmental Performance Reports (DPRs) provide a source of budgetary, expenditure and variance information based on the PAA. In the case of the EPHP, information is available for the two fiscal years (2013-14, 2014-15) for which the program has not been included in broader reported activities. In the 2014-15 DPR, the program spent approximately $24.5 million less than originally anticipated ($59.6 million) stating that the variance between actualEndnote iv and planned spending was mainly due to the reallocation of funding to sub-sub program Clinical and Client Care for continuity of access of nursing services in remote and isolated First Nations communities, which was partly offset by additional funding received through Supplementary Estimates to support the delivery of the First Nations Water and Wastewater Action Plan. However, it should be noted that, while there was a budgetary variance, more human resources (150 FTEs) were attributed to the program than originally planned (132 FTEs), mainly due to a realignment of resources from plans in order to meet program needs.

Similarly in 2013-14, the program spent just under $6.6 million less than anticipated ($82.4 million), noting that the variance that year was primarily due to reductions associated with the realignment of resources for the implementation of the B.C. Tripartite initiative and a reallocation of funding within this strategic outcome to Clinical and Client Care to address needs and priorities. Similar to the 2014-15 fiscal year, more human resources were dedicated to these activities than originally planned.

EPHP expenditures are captured by an overall funding source. For example, an EHO's salary may be funded with FNWWAP resources (water and wastewater) whereas her annual activities may actually cover other various areas of programming (e.g., communicable disease control, solid waste, etc.) depending on community priorities. This approach to budgeting and assignment of expenditures limits the extent to which the evaluation can quantitatively assess the economy and efficiency of the EPHP from a financial data perspective. The evaluation has collected more qualitative information on stakeholders' perceptions of economy and efficiency, along with some potential areas for improvement which are presented in this section.

Economy

The EPHP has continuously worked with First Nations communities and organizations to offer the most useful, relevant services, policies and knowledge transfer in the various areas of environmental public health. The evaluation found that First Nations communities and organizations are satisfied with the services from the EPHP, indicating that the program is expending resources on areas that are addressing key First Nations priorities and that are relevant. This is a good indicator of the economy of the program. An additional area of economy noted by the evaluation is the strong network of EHOs that can provide surge capacity, share expertise and resources.

Many representatives from First Nations communities and organizations and representatives from EPHP noted that the relationship between the EHOs and community is an important aspect of the economy of the program delivery. The EHO investment in community work plans and adjusting the services from the EPHP to meet the needs at a community level provides some assurance that the services being offered are tailored according to community priorities and needs. This planning and flexibility in programming promotes the deployment of resources based on the priorities of the community and the EPH concerns of the local area.

While the flexibility of the program and services offered was highlighted as a positive aspect in many interviews, there were a few EPHP representatives that indicated there may be a need to determine whether activities do actually align with the program's core mandate and eliminate areas of potential duplication. The two main areas cited as examples included clarification of funding of drinking water sampling at the water plant level in some communities (e.g., role of EHO, CBWM, or plant operator), and the extensive involvement in food safety training to determine whether it is verging on job skills training more than addressing environmental public health risks.

The strong network of EHOs in regions was identified as supporting economic program delivery for the EPHP. The network allows for quick sharing of expertise and resources across EHOs and communities, along with a "surge capacity" to cover emergencies and need for quick response from EHOs. This need to quickly respond to emergency situations such as flooding in First Nations communities is expected to continue to grow with anticipated impacts of climate change.

Efficiency

Already considered a "lean" program, there have been a number of efficiencies developed over the past five years. Changes in approaches to budgeting and funding at the regional level make it challenging to identify national level efficiencies for the EPHP. The workload of some EHOs may be unsustainable in the longer term, potentially impacting on the effectiveness of the EPHP. There is a strong likelihood that there will be significantly greater demands placed on the EPHP given the increased infrastructure changes, ongoing growth and development of First Nations communities as they address EPH risks in various areas. Given the variability in delivery of the EPHP across regions, and differences in capacity and priorities between First Nations communities, these challenges in sustainability and its impact on effectiveness may be more marked in some areas compared with others.

As illustrated in Figure 4, the majority of EHOs surveyed indicated that the efficiency of their work methods had increased over the past five years (62%). As well, the majority also indicated that the quality and level of service they can provide to meet community needs had increased during the same period (56%). These are similar levels of increased efficiency noted by EHOs in 2011 (67%), and increased quality and level of service (63%).

Figure 4: Efficiency and quality of service improvements in past five years

Figure 4A: Efficiency of EHO's work methods
Figure 4A
Figure 4A - Text Description

This figure shows the efficiency of EHO's work method. Among the survey respondents, 21% stated that the efficiency increased significantly, 41% started that the efficiency somewhat increased, 22% stated that the efficiency stayed the same, 13% stated that the efficiency somewhat decreased and 4% of them stated that it decreased significantly.

Figure 4B: Quality & level of service to communities
Figure 4B
Figure 4B - Text Description

This figure shows the quality and level of services to communities. Among survey respondents, 16% stated that the quality and level increased significantly, 40% stated that the quality and level were somewhat increased, 32% stated that the quality and level stayed the same, 11% stated that the quality and level somewhat decreased, and 2% stated that the quality and levels decreased significantly.

Source: EHO Survey, 2016 (n=63).

In terms of efficiency, many program representatives, particularly in the regions, characterized EPHP as a lean program with a few believing that the program is actually underfunded. Many regional representatives noted that the workload and activities of EHOs are highly monitored for efficiency. Some regional representatives reported that they are experiencing significant workload pressures. Many regional representatives noted that workload issues are likely to get increasingly challenging as a key cost driver for the program is the number and profile of communities and their priorities. It was noted among regional representatives and other federal government departments interviewed that many First Nations communities are growing and there are consistently more complicated commercial and community infrastructure being constructed. Budget 2016 also committed to substantial increased investments for First Nations community infrastructure projects. In addition, according to interviews with program representatives and First Nations' community representatives, there is increasing emphasis on EPH risks within First Nations communities and additional funding being allocated to address some of these issues. All of these circumstances combined with potential continued work in the development of drinking water and wastewater regulations point to future increased demands that are likely to be placed on the EPHP.

A draft report from the Workload Assessment Working Group found that there are significant shortages of EHOs when workloads were assessed against requirements in meeting the minimum standards set out in the National Framework for the Environmental Public Health Program in First Nations Communities South of 60. The report noted that this shortage could be impacted by various factors, such as time limited FNWWAP contributions (currently funding approximately 40% of EHOs); possible expansion of EHO duties due to environmental changes; the development of a regulatory framework for water and wastewater; and, other human resources factors that are common across the federal government (e.g., anticipated retirements).

Interviews with HQ representatives noted some concerns with respect to their oversight role in understanding and reporting on EPHP efficiencies at a national level given the changes in budgeting and allocation approach (i.e., FNIHB's approach to zero-based budgeting using regional management operational plans) combined with the shift of some administrative responsibilities to the regions. Some regional representatives indicated that the changes in budgeting approach and shift of administrative responsibilities could have created inefficiencies for the EPHP at the regional level as they are required to perform more administrative aspects of program delivery.

Also potentially impacting efficiency is the finding from the key informant interviews with EPHP representatives regarding a perceived lack of clarity of roles and responsibilities between HQ and some regions (as well as provincial health authorities). According to some HQ and regional representatives, there is no clear definition or direction at this point with respect to leadership and responsibilities. Regional representatives varied considerably in their characterization of the relationship between HQ and regions as ranging from "clear and functional" on the one hand to "strained" on the other. It was noted in interviews that regions and HQ have different reporting needs that can lead to different perspectives on program priorities (e.g., changes to the data systems to meet the national reporting needs of HQ versus the more immediate delivery objectives of regions). For example, having all Regions on one database/information system would, from HQ's perspective, simplify and improve efficiency and accuracy of reporting on national performance and public health indicators. However, some Regions prefer different databases because these better suit the needs and practices in place in their Regions. Many HQ and regional key informants noted that until recently (March 2016), there have been no opportunities to meet in-person, leading to an erosion of peer relationships given less direct, ongoing contact between regions and HQ.

Performance Measurement

There have been improvements in data collection and quality control over the past five years; however there remain challenges with the comprehensiveness and utility of data collected through the various administrative and monitoring systems.

EPHP has prepared an annual performance report on various national-level indicators for four of the five years within the evaluation timeframe. In 2012-13, FNIHB finalized the EPH Indicator Reporting Strategy with the goal of increasing the evidence base on EPH conditions and potential risks in First Nations communities on reserve. This strategy was implemented for the 2012-13 and 2013-14 annual reports. From a review of documents, the annual reporting for 2014-15 appears to have reverted back to be more closely aligned with specific outputs/activities according to the logic model developed for the EPHP (similar to 2011-12).

Reports from key informants, which were confirmed with the review of performance data, indicate that there have been some improvements in data collection and quality control over the past five years with respect to EHO activities in communities. There remain challenges with comprehensiveness and utility of data collected through the various administrative and monitoring systems (outside of drinking water monitoring), with reporting from these systems having been variable over the period covered by the evaluation.

Many program representatives interviewed said that there should be increased efforts in improving performance data. The evaluation found that areas cited that could be improved included:

  • addressing data gaps with contributors (e.g., INAC, First Nations communities that have entered into a Health Funding agreement);
  • placing more emphasis on results in addition to outputs that would support national reporting on outcomes (particularly related to recommendations);
  • structure data systems so that they allow for data mining capabilities and potentially linking to other data sources such as GIS – this would make the data more useful in answering questions, planning and allocating resources; and
  • ensuring that there is capacity within data systems for a regional/community lens to support a more proactive and risk-based service delivery approach.

5.0 Conclusions

5.1 Relevance Conclusions

5.1.1 Continued Need

The EPHP is designed to address a continued and growing need among many First Nations communities to identify, address and prevent human health risks associated with exposure to hazards within the natural and built environments. Environmental public health is an essential component of public health systems, and is a key contributor to improving health and well-being of populations. As was identified in the previous evaluation, many First Nations communities continue to experience significantly challenging conditions with respect to environmental health risks compared with many other Canadian communities. These ongoing challenges result from issues such as sub-standard housing and living conditions, poor quality or maximum capacity water and wastewater systems, lack of certified water plant operators, climate and geography, and numerous socio-economic inequalities.

5.1.2 Alignment with Government Priorities

As was found in the previous evaluation, the EPHP is designed to align with federal priorities and strategic outcomes sought for First Nations communities. Furthermore, the EPHP is aligned with Health Canada's strategic outcome of First Nations and Inuit communities and individuals receiving health services and benefits that are responsive to their needs so as to improve their health status.

5.1.3 Alignment with Federal Roles and Responsibilities

Since the previous evaluation, the Government has passed the Safe Drinking Water for First Nations Act (2013). The introduction of this Act moves the federal role in EPH risk mitigation beyond advisory to include a possible regulatory role. During this evaluation time period, safe drinking water regulations for First Nations were under development; however, as of 2016, the Act became subject to review by INAC Ministerial direction, and regulatory development was put on hold. Health Canada continues, therefore, to play an advisory role which varies in each region in order to meet specific community needs and priorities.

As different federal departments have a role to play in supporting First Nations communities in addressing EPH risks, it is necessary that the contribution of all federal departments is coordinated and aligned to maximize effectiveness and achieve common strategic outcomes.

5.2 Performance Conclusions

5.2.1 Achievement of Expected Outcomes (Effectiveness)

The EPHP has clearly demonstrated progress in the achievement of immediate outcomes over the past five years. The findings indicate that there have been further improvements in timely data collection, the results from which are then disseminated at various levels either through recommendations or other types of reporting; an increased use of evidence to guide EPH activities with relevant research being conducted in communities to address EPH priorities; and an overall increased awareness of EPH risks among First Nations communities. This awareness has contributed to an increased use of and demand for EPHP services in communities.

Similar to the findings from the previous evaluation, although progress on immediate EPHP outcomes is quite clear, the extent to which intermediate outcomes have been achieved is less clear. The steps required to mitigate an EPH risk are generally to initially identify, then assess, and then address (if required from the assessment). The evaluation findings indicate that there has been continued progress over the past five years with respect to First Nations communities capacity to undertake the first two steps (identify and assess) EPH risks, but less progress with undertaking activities required to address the EPH risks. While there are some areas where this third step is occurring (e.g., drinking water advisories, food safety issues), overall there remain challenges for communities to address the more complex EPH risks.

The role of the EPHP is to assist communities by providing training and education around EPH risks according to community priorities, developing recommendations for addressing EPH risk based on investigations, and reviewing infrastructure plans from a public health engineering perspective. Given this role, EPHP has the capacity to undertake the first two steps to identify and to assess EPH risks that often result in recommendations for a First Nations community. There is neither funding directly attached to the EPHP nor a mandate to actually undertake the third step of addressing an EPH risk, based on the EPHP recommendation.

The contingencies and factors involved in actually achieving the third step in the outcome of addressing EPH risks are numerous including community priorities and resource availability, all of which are external to the EPHP. This indicates that if the EPHP is expected to contribute to communities addressing their more complex EPH risks, the program will continue to have to coordinate and collaborate with the multitude of other stakeholders involved that assist First Nations communities in determining their priorities and obtaining resources to address EPH risks.

While the program logic would indicate that EPHP's progress in the achievement of immediate outcomes combined with the partial progress in achievement of intermediate outcomes should be contributing to the longer term outcome of improved health and well-being of First Nations, the evaluation was unable to actually assess this link accurately. From a theoretical perspective, this contribution would be occurring to the extent that EPH risks are actually being addressed in First Nations communities.

5.2.2 Demonstration of Economy and Efficiency

The EPHP demonstrates some aspects of economy and efficiency in its delivery. Given the manner in which budgets are developed and resources allocated, it is challenging to determine the extent to which planning and budgets align with actual expenditures in each of the program's regions and activity areas. This also contributes to challenges with calculating some measures of efficiency using the financial data available for the program such as ratios of cost per output.

The program's strong, positive connections with First Nations communities and organizations, particularly at the community level, assist in ensuring the relevancy of the EPHP's services and advice. The EPHP is expending resources on EPH activities that are contributing to outcomes important to First Nations communities. There were demonstrated efficiencies in the past five years for the program, many of which were highlighted in the areas of service delivery at the regional levels.

A key component of the success of the EPHP is the EHO. While there were numerous areas of efficiency outlined for EHOs, there is a concern that the sustainability of their effectiveness may be at risk given the increasing workload in many of the regions, and the anticipated significantly greater demands being place on the EPHP due to increased awareness of EPH risks in many communities, increased funding for community infrastructure projects, and the ongoing growth and development of First Nations communities as they address EPH risks in various areas. The impact of workload pressures among EHOs is somewhat varied by region due to differences in capacity and priorities between First Nations communities.

Highlighted as an area for improvement in the previous evaluation, the performance measurement for the EPHP has improved during the past five years. Attempts were made at moving the reporting from strictly counting of outputs more into the realm of outcomes. In addition, there was work on the development of some of the data collection systems. Despite these improvements, there remain some challenges with the comprehensiveness and utility of data collected.

6.0 Recommendations

Recommendation 1

Explore options to enhance collaboration with stakeholders to support First Nations in addressing and mitigating complex environmental health risks (e.g., those that may require longer-term and more costly interventions). While there are many coordination and collaboration activities underway already, it is clear that for the intermediate (and thus longer term) outcomes to be achieved there needs to be continued involvement by EPHP in contributing evidence and assessments to support or promote the mitigation of complex environmental public health risks. In order to better assess the degree to which outcomes are achieved, the impacts of EPH collaboration and coordination on EPH risk mitigation should be documented, with the understanding that whether such risks are actually mitigated is often outside the EPHP's control.

There are numerous other stakeholders involved in various areas of EPH in First Nations communities. These stakeholders include: First Nations communities, various First Nations organizations, other federal government departments and agencies (e.g., INAC, EC, PHAC, CFIA, CMHC), various branches within Health Canada, and provincial governments. INAC and Health Canada require collaboration specifically in a number of areas associated with EPH given their combined contributions to achievement of EPHP objectives. For example, while INAC is the lead on the First Nations Water and Wastewater Action Plan, Health Canada has important supporting roles. Similarly, investigations by EHOs or reviews by public health engineers can result in recommendations for infrastructure improvements that are being funded through INAC programming. EPHP is only one of the players in this complex matrix; however, the EPHP is well-positioned to continue to provide evidence and information required to promote the mitigation of complex EPH risks, given their expertise and strong working relationships and connections to the community. EPHP's coordination and collaboration activities should continue at various levels including community, sub-region, region and national levels.

Recommendation 2

Review the current EHO allocation and workload within and across regions to determine sustainability of service to communities considering various forecasted additional demands and complexities. Building on the findings from the workload analyses currently underway, it will be important for the EPHP to clearly understand the challenges involved resulting from the current EHO allocation and workloads at both regional and national levels. This review would need to take into account the anticipated additional demands and complexities for many First Nation communities as a result of various changing contexts (e.g., increased infrastructure funding, climate change, economic development, population growth).

Recommendation 3

Produce an assessment of the impact of the recommendations made by EHOs to Chiefs and Councils in the area of food, water and housing. Given that the recommendations that EHOs develop are an essential output to achieving some of the desirable outcomes for the program, they are potentially a rich source of information and data for various uses at multiple levels. This would likely enhance the utility of the performance and monitoring data being collected by the program.

Appendix 1 – Core Areas of Program Activity Focus

  • Drinking Water - The EPHP provides public education about safe drinking water and risk prevention; training and education material to Community-Based Drinking Water Quality Monitors (CBWMs)Footnote 26 ; conducts monitoring and testing of drinking water supplies including interpretation of results, recommendations for corrective action and mitigation (including issuance and rescindence of drinking water advisories); and provides engineering reviews of water infrastructure project proposals from a public health perspective.
  • Wastewater - The EPHP identifies existing and potential hazards associated with wastewater disposal in order to reduce and prevent public health risks. Program activities focus on community wastewater treatment plants as well as on-site sewage disposal systems. Activities include conducting environmental public health assessments, providing public education, and providing engineering reviews of wastewater infrastructure project proposals from a public health perspective.
  • Health and Housing - The EPHP works with First Nations communities and other stakeholders to address public health issues at the various stages of housing: site and design, construction, occupancy and demolition. This is accomplished through on-request public health inspections of housing, public education and training sessions.
  • Solid Waste Disposal - The EPHP works with communities and other agencies to help limit public health risks posed by solid waste disposal. Activities include conducting environmental public health assessments of disposal sites and transfer stations, and providing advice and public education about health waste disposal practices, and providing engineering reviews of solid waste site project proposals from a public health perspective.
  • Food Safety – The EPHP works with First Nations communities to prevent foodborne illness and address public health issues related to both traditional and conventional foods. Activities include public education, food handler training and routine and on-request public health inspections of permanent, seasonal and special event food service facilities.
  • Facilities Inspections – The EPHP works with First Nations communities, owners, operators, employees and users of facilities (health, community care, recreational and general facilities) to help prevent the spread of communicable disease, minimize public health risks and reduce safety hazards. Activities include providing routine and on-request inspections, providing advice, guidance and recommendations and delivering public education and awareness sessions related to public health and safety within facilities.
  • Environmental Communicable Disease Control - All EPHP activities aim to prevent illness and the spread of communicable diseases (e.g., monitoring drinking water supplies, providing food facility inspections and regular inspections of other public facilities). Specific activities, such as, inspections, outbreak investigations, surveillance and public education are also undertaken to prevent and control foodborne, waterborne, and vectorborne environmental communicable diseases. All activities are carried out in collaboration with local, regional, provincial and/or national communicable disease staff. Note that as of 2015-16, this program area within the National Office has been transferred to a different unit within Health Canada. However, the work of EHOs are environmental public health staff at the regional and community level remains the same.
  • Emergency Preparedness and Response - The EPHP works with communities and stakeholders to support environmental public health considerations in emergency planning, response and recovery activities, such as: flooding and forest fires. Activities include assessment of environmental public health risks during emergency planning, response and recovery situations and providing advice, guidance, and recommendations on how to minimize these risks. Note that as of 2015-16, this program area within the National Office has been transferred from the EPHP to a different unit within Health Canada. However, the work of EHOs and environmental public health staff at the regional and community levels remains the same.

Appendix 2 - Logic Model

Appendix 2
Appendix 2 - Text Description

Key programming under Health Canada's First Nations and Inuit Health Branch's Environmental Public Health Program continues to include activities that focus on two core areas: Public Health Assessments; and Research and Surveillance. Public health assessments include Drinking Water; Wastewater; Solid Waste Disposal; Food Safety; Housing; Facilities Inspections; Environmental Communicable Disease Control; and, Emergency Preparedness and Response. Research and Surveillance includes Environmental Contaminants and Laboratory.

The Health Canada's First Nations and Inuit Health Branch's Environmental Public Health Program supports a continued and growing need among many First Nations communities program to identify, assess and address human health risks associated with exposure to hazards within the natural and built through the funding of the following activities: environmental public health risk assessment; research and surveillance; training, public education, and awareness; collaborative relationship building.

As a result of each activity, Health Canada's First Nations and Inuit Health Branch's Environmental Public Health Program activities generate a number of outputs, namely:

  • Inspection, engineering reviews
  • Research dissemination
  • Training sessions, awareness activities
  • Joint projects, agreements

Through these outputs, program activities correspond with specific immediate, intermediate and long-term outcomes. In the immediate term, program activities are expected to lead to improved environmental public health risk identification, specifically: Timely data collection, analysis, dissemination and reporting (1); Evidence-based decision-making (2); First Nations and Inuit awareness of environmental public health risks (3); and Immediate risk mitigation activities (4).

The achievement of these immediate outcomes is expected to lead to the following intermediate outcomes: Increased capacity of First Nations and Inuit individuals and communities to address environmental public health risks through Individual risk reduction practices (1); and Community engagement and actions towards environmental public health risk reduction (2). In the longer-term, the Health Canada's First Nations and Inuit Health Branch's Environmental Public Health Program activities expect to lead to an improved health and well-being of First Nations and Inuit individuals and communities.

Appendix 3 - Summary of Findings

Appendix 3
Appendix 3 - Text Description

This appendix shows the summary findings done on activities, outputs, immediate outcomes, intermediate outcomes and long-term outcome. The activities, outputs and long-term outcome were not assessed. Regarding the immediate outcome "improved public health risk identification", there was success in evidence based decision making, awareness of enviro public health risk, immediate risk mitigation, but indicates that there were success and some minor but important issues to address in timely data collection, analysis, dissemination and reporting. It also shows that regarding the intermediate outcome "Increased capacity of First nations and Inuit individuals and communities to address environmental public health risk" there were success and some minor but important issues in individual risk reduction practices and community engagement and action toward enviro public health risk reduction.

Appendix 4 – Evaluation Matrix

Appendix 4 – Evaluation Matrix
Evaluation Issues Evaluation Questions Indicators Methods / Data Sources
Secondary Data Sources Primary Data Sources
Relevance

Issue #1: Continued need for the Program

1.1 What are the current and projected environmental health risks facing First Nations communities?

  • Evidence of current or emerging environmental health risks with a focus on First Nations communities.
  • Perception of current and projected environmental health risks with a focus on First Nations communities, and identification of changes since 2010 Evaluation.

Literature Review

scholarly articles , gray literature

Document/File Review

surveillance and Statistics Canada reports

Comparative Analysis

EHO Web Survey

Issue #2:
Alignment with government and departmental priorities

2.1 What are the federal priorities related to First Nations environmental public health?

  • Identification of federal priorities.
  • Program activities correspond to recent/current federal priorities.

Document/File Review

Government of Canada documents: Speech from the Throne, Budget, National and/or international agreements

-

2.2 What are the Health Canada priorities related to First Nations environmental public health?

  • Identification of departmental priorities.
  • Program activities aligned with, and contribute to, departmental priorities .

Document/File Review

  • Departmental documents: RPP, DPR
  • Branch documents: Strategic Plan, presentations to key management tables
  • operational plans, workplans, annual reports
-

Issue #3: Alignment with federal roles and responsibilities

3.1 What is the federal role related to environmental public health of First Nations? Is the federal role aligned with current activities?

  • Identification of federal and departmental role.
  • Program activities align with federal jurisdiction, departmental mandate and roles.
  • Evidence of extent to which Health Canada is meeting international commitments.

Document/File Review

  • legal authorities: Health Canada Act, Federal Indian Health Policy, Safe Drinking Water for First Nations Act
  • policy and program authorities (e.g., FNWAAP, Clean Air, etc.)
  • other obligations (e.g., MOU for First Nations under contribution agreements)
  • operational plans, workplans, annual reports
-

3.2 Does the federal role duplicate or complement the role of other stakeholders? Are there any gaps or overlaps?

  • Presence/absence of other programs that complement or duplicate objectives of program.
  • Views on programs that complement, overlap or duplicate Health Canada involvement.

Document/File Review

program documents (TBD)

Key Informant Interviews

  • internal: HC staff
  • external: OGDs, funding recipients

EHO Web Survey

Performance

Issue #4: Achievement of Expected Outcomes

4.1 To what extent has environmental public health risk identification improved?

  • Summarize indicators found under 4.1 a) to 4.1 d)
- -

4.1 a) Timely data collection, analysis, dissemination and reporting

  • Views on achievement of this outcome, identification of potential barriers, progress from 2010 Evaluation.
  • Performance data on type of data systems available (e.g., early warning databases) and identification of the use of those systems, presence of common indicators, list of tools, procedures, guides, training and research provided and/or developed, number/percentage of staffed EHO position, number/percentage of communities with access to a trained CBWM or EHO, number of water samples taken and analyzed, data on water systems with annual sampling of recommended core chemical parameters.

Document/File Review

program files

PM Data Review

annual reports, DPR, internal databases

Key Informant Interviews

  • internal: HC staff
  • external: Funding recipients

EHO Web Survey

4.1 b) Evidence-based decision-making

  • Views on achievement of this outcome, identification of potential barriers, progress from 2010 Evaluation.
  • Identification of research funded, dissemination activities, uptake/use of research findings/deliverables
  • # of community-based/participatory research reports on environmental health hazards that are available (DPR 2013-14).

Document/File Review

program/project files

PM Data Review

annual reports, DPR, internal databases

Key Informant Interviews

  • internal: HC staff
  • external: OGDs, funding recipients

EHO Web Survey

4.1 c) First Nations population awareness of environmental public health risks

  • Views on achievement of this outcome, identification of potential barriers, progress from 2010 Evaluation.
  • Availability of public/community education and awareness materials/strategies
  • Access to, and usage of, environmental public health services.

Document/File Review

program/project files, public opinion research

PM Data Review

annual reports, DPR, internal databases

Key Informant Interviews

  • internal: HC staff
  • external: Funding recipients

EHO Web Survey

4.1 d) Immediate risk mitigation activities

  • Views on achievement of this outcome, identification of potential barriers, progress from 2010 Evaluation.
  • Performance data on risk mitigation activities undertaken based on environmental public health activities results (if available).

Document/File Review

program/project files

PM Data Review

annual reports, DPR, internal databases

Key Informant Interviews

  • internal: HC staff
  • external: OGDs, funding recipients

EHO Web Survey

4.2 To what extent First Nations individuals and communities have the capacity to address public health risks?

  • Summarize indicators 4.2 a) and 4.2 b)
- -

4.2 a) Individual risk reduction practices

  • Views on achievement of this outcome, identification of potential barriers, progress from 2010 Evaluation.
  • Performance data on inspection activities (number/percentage of home inspections, degree to which inspection recommendations implemented / measures taken to address EHOs' recommendations), trends in confirmed waterborne, foodborne diseases and outbreaks.
  • Number of applications to AANDC to request support to remediate/mitigate EPH risks.

Document/File Review

Internal files (other than EHIS database)

PM Data Review

annual reports, DPR, internal databases

Key Informant Interviews

  • internal: HC staff
  • external: Funding recipients

EHO Web Survey

4.2 b) Community engagement and actions towards environmental public health risk reduction

  • Views on achievement of this outcome, identification of potential barriers, progress from 2010 Evaluation.
  • Alignment between community capacity building dimensions and program activities (i.e., evidence of how activities conducted led to increase community engagement and actions, including participation, leadership, community structures, external supports, asking why, obtaining resources, and skills, knowledge, and learning, linking with others, sense of community) as per community capacity building tool categories.
  • Performance data on:
    • data on water and wastewater project proposals reviewed by the program .
    • % of First Nations communities with integrated Pandemic Preparedness/ Response Plans and Emergency Preparedness/ Readiness Plans.
    • % of First Nations communities that have access to a trained Community-based Drinking Water Quality Monitor or an Environmental Health Officer to monitor their drinking water quality. (RPP 2014-15).

Document/File Review

materials related to training, public education, MOAs/MOUs, meetings, joint projects, workplans, decks, etc.

PM Data Review

annual reports, DPR, internal databases

Key Informant Interviews

  • internal: HC staff
  • external: Funding recipients

EHO Web Survey

4.3 To what extent has the health and well-being of First Nations individuals and communities improved?

  • Views on achievement of this outcome, identification of potential barriers, progress from 2010 Evaluation.
  • Evidence of public opinion research on the confidence in the safety of drinking water and other EPH risks.
  • Performance data on trends in drinking water advisories, data on homes having access to piped water, proportion of on-reserve public water systems that met weekly national testing guidelines for bacteriological parameters (based on testing frequency recommended in the Guidelines for Canadian Drinking Water Quality (RPP 2014-15) .

Literature Review

Document/File Review

Public Opinion Research,

PM Data Review

annual reports, DPR, internal databases

Key Informant Interviews

  • internal: HC staff
  • external: OGDs, funding recipients

EHO Web Survey

4.4 How successful was the program in responding to current needs and priorities?

  • Evidence of the program responding to current needs and priorities, including carrying out activities in higher risk areas.
  • Views on ability to respond to needs and priorities, identification of potential barriers, progress from 2010 Evaluation.

Document/File Review

strategic plans, operational plans, workplans

PM Data Review

annual reports, DPR, internal databases

Comparative Analysis

Key Informant Interviews

  • internal: HC staff
  • external: OGDs, funding recipients

EHO Web Survey

Issue #5: Demonstration of Efficiency and Economy

5.1 Is the program delivered in an efficient manner? How and in what ways can efficiency be improved?

  • Qualitative evidence that Health Canada has structures/mechanisms in place, so that the most efficient means are being used to administer/deliver the program. For example:
    • Governance processes (clear delineation of roles and responsibilities, minimize duplication/overlap.
    • Priority setting/planning (streamlined planning and decision-making, use of risk management).
    • Timeliness of delivery (e.g. routine vs. emergency requests).
    • Funding recipients and program management identify and take into consideration their own best practices/lessons learned in improving alternative approaches.
    • Whether budgetary decisions impacted the efficient delivery of program activities.

Document/File Review

strategic plans, operational plans, workplans, Committee Terms of References, risk management plans, program decks/Memos on financial decisions

Financial Data Review

Comparative Analysis

PM Data Review

annual reports, internal databases

Key Informant Interviews

  • internal: HC staff
  • external: Funding recipients

EHO Web Survey

5.2 Has the program produced its outputs and achieved its outcomes in the most economical manner? How and in what ways can economy be improved?

  • Variance between planned and actual expenditures, and implications.
  • Views on whether :
    • funds are appropriately targeted,
    • costs of producing outputs is as low as possible and value is being obtained,
    • budgetary decisions impacted the economical delivery of program activities,
    • alternative program models that would achieve outcomes at lower cost (if/where available).
  • Appropriateness of administrative overhead % (if/where relevant).
  • Degree of leverage (if/where relevant).
  • Comparison of delivery cost per output between similar programs/projects (if/where available).

Document/File Review

strategic plans, operational plans, workplans, program decks/Memos on financial decisions

Financial Data Review

CFOB to provide planned and actual spending, validated with program leads

Key Informant Interviews

  • internal: HC staff
  • external: Funding recipients

EHO Web Survey

5.3 How is performance measurement being used?

  • Existence of logic model, performance measurement framework or strategy.
  • Adequate collection of performance information.
  • Use of performance information in decision-making.

Document/File Review

internal files, including Memos, Records of Decision, etc.

PM Data Review

PM strategy, internal databases, annual reports, RPP, DPR

Key Informant Interviews

internal: HC staff

EHO Web Survey

Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: