Evaluation of the Addressing Air Pollution Horizontal Initiative

Acknowledgements

The Evaluation Project Team would like to thank the individuals who contributed to this project. In particular, we would like to acknowledge the contribution of program representatives from Environment and Climate Change Canada, Health Canada and the National Research Council Canada who provided input and comments to inform the findings and recommendations of this evaluation.

Executive summary

This report presents the findings from the evaluation of the Addressing Air Pollution Horizontal Initiative (AAPHI), the cornerstone of the federal government’s approach to addressing air pollution. The AAPHI is led by Environment and Climate Change Canada (ECCC) in collaboration with Health Canada (HC) and the National Research Council (NRC). It aims to improve air quality, reduce impacts of air pollution on health and the environment and provide Canadians with the tools to make informed decisions to reduce their exposure to indoor and outdoor air pollutants. Core activities include science and reporting; policy and analysis, mitigation actions for outdoor air quality and indoor air quality activities. ECCC, HC and NRC received about $90 million per year for AAPHI activities between fiscal year (FY) 2016 to 2017 and FY 2019 to 2020.

The evaluation focused on the period from FY 2016 to 2017 to the middle of FY 2019 to 2020 and included all AAPHI activities except HC’s work under the National Radon Program, which will be assessed as part of separate evaluation activities. The evaluation examined questions related to relevance, efficiency and effectiveness (as per the 2016 Treasury Board Policy on Results). Lines of evidence included document and data review, key informant interviews and focus groups.

Findings and conclusions

Relevance

Canada has a low level of ambient air pollution compared to some regions of the world and air quality in the country has generally improved over time. However, available scientific evidence, along with the potential implications of climate change and other emerging issues for air pollution and air quality, underscore the need for ongoing effort.

The AAPHI aligns with federal roles and responsibilities. All stakeholders agreed that the federal government’s role in addressing air pollution and air quality should continue. Both industry and provincial/territorial (P/T) participants in the evaluation emphasized the importance of avoiding duplication with P/T regulatory activities. P/T representatives urged the federal government to clearly articulate and share with the P/T its long-term “master plan” or agenda for air quality. This would help them in planning their own activities, avoid duplication and minimize regulatory and reporting impact on industry.

Although the AAPHI aligns with current federal priorities, there is considerable support among internal and external participants in the evaluation for enhancing its alignment with the government’s climate change and Indigenous reconciliation priorities.

Effectiveness

Key accomplishments of the AAPHI during the period covered by this evaluation include:

• the ongoing implementation of the Air Quality Health Index (AQHI) across Canada
• the development of new Canadian Ambient Air Quality Standards (CAAQS) for sulphur dioxide and nitrogen dioxide and revised CAAQS for ozone
• the implementation of the Multi-Sector Air Pollutants Regulations and a number of non-regulatory instruments establishing base-level industrial emission requirements (BLIERs) for several pollutants and sectors
• the ratification of the 1999 Protocol to Abate Acidification, Eutrophication and Ground-level Ozone (the Gothenburg Protocol) and its 2012 amendments

However, some planned mitigation measures for outdoor air have not advanced to the extent originally envisioned. These include the establishment, through regulatory or non-regulatory instruments, of BLIERs for a number of sectors, in particular those where consensus was not reached during the Air Quality Management System (AQMS) development process. Progress has been made in establishing BLIERs, including through the Multi-Sector Air Pollutants Regulations and a number of non-regulatory measures. Regulations to limit some volatile organic compounds (VOCs) from the petroleum refining sector establish some BLIERs for the refining sector, but planned regulations to reduce emissions of pollutants other than VOCs from the petroleum refining sector have not been brought forward. There was an expectation that the federal government would move forward appropriately to establish BLIERs where no consensus was reached. To date, this has not happened. However, many measures aimed at addressing climate change are expected to reduce air pollutant emissions in several sectors; there is a need to reconfirm the extent to which measures to address the air pollutants and industrial processes targeted by the BLIERs are still needed in light of these measures.

The need for action in relation to petroleum refineries is supported by data indicating that Canadian refineries emit far more pollutants than their counterparts in the United States and that one or more of the CAAQS for sulphur dioxide, nitrogen dioxide, ozone and fine particulate matter is being exceeded in seven out of 13 areas where there are petroleum refineries and oil sands upgraders. The Government of Canada indicated that it had begun to develop regulations to address these emissions in its 2018 response to Parliament on the review of the Canadian Environmental Protection Act, 1999.

The evaluation found evidence of progress toward some of the expected results of the AAPHI, although in some cases, evidence is lacking to support firm conclusions.

• Access to information and analysis. A considerable volume of information and analysis on air quality and air pollution is produced and used to inform decision making by a range of internal and external stakeholders. There was general agreement on the high quality of the information and analysis available. Perceived weaknesses include key studies and reports not being well publicized, information not being consolidated in one easily accessible location and websites and reports not being updated in a timely fashion. ECCC developed but has not yet implemented a National Air Quality Communication Strategy, with recommendations for enhancing communication to various target audiences
• Increased public awareness and behaviour change. The AQHI is the federal government’s main mechanism for disseminating information about outdoor air quality and health risks to the Canadian public. External participants in the evaluation praised it for its simple messaging and standardized approach to air quality reporting. However, the most recent national survey, from 2017, indicates that few Canadians are aware of it or are using it to guide their behaviour. With regard to indoor air, HC has limited information on the extent to which the public is aware of and using the information it generates. NRC’s indoor air research and evaluation activities have addressed the needs of its clients in various ways and informed their decision making. Some NRC evaluated solutions and technologies have experienced market uptake.
• Compliance with air pollution measures. Overall, there is limited information on the extent to which regulated communities comply with mitigation measures for air pollution. The program has developed performance indicators for recently introduced mitigation measures and compliance data will be reported in the coming years.
• Reduced emissions. Emissions of air pollutants from sources subject to mitigation measures have mainly declined over the last 15 years. However, some emission indicators show evidence of recent increases.
• Clean air and reduced adverse impacts. While a majority of Canadians live in areas where air quality standards are achieved, 25% do not. Furthermore, 30% of P/T air zones did not meet one or more of the 2015 CAAQS for fine particulate matter, ozone, or both. With regard to adverse health impacts, although mortality risk due to air pollution has declined slightly in recent years, the number of Canadians who die from air pollution each year remains significant and air pollution causes a variety of morbidity outcomes. Thus, Canadians continue to be exposed to levels of air pollution that pose risks to their health. For risks to ecosystems, the program has recently developed an acidification indicator and plans to report on it in the future.

Efficiency

Internal and external participants in the evaluation consider the overall design of the AAPHI to be sound. Perceived strengths include its collaborative approach to delivering key activities, its robust research, science and technical expertise and capacity, its standardized approach to air quality monitoring and reporting and its focus on transparency and reporting to the public. They also saw Canada’s increased profile in recent years as an international leader with respect to air quality research and management as an important strength. International stakeholders praised Canada for its contributions on the international stage and encouraged it to continue playing a leadership role.

Several possible improvements to program design emerged from this evaluation.

• Better integration between air quality and climate change policy.Internal and external participants in the evaluation emphasized the need to more systematically consider and account for the inter-relationships between air quality and climate change, from both a scientific and a risk management perspective, while maintaining these as distinct policy areas. They identified both a need to better understand the impacts of climate change on air quality and increased air pollution on climate change as well as a need to consider the potential impacts of climate change actions on air pollutants and actions to address air pollution on greenhouse gases. As well, they noted that work is required to identify ways to minimize negative impacts and maximize co-benefits, to optimize risk management strategies.
• Greater focus on Indigenous peoples. Evidence indicates that Indigenous peoples are particularly vulnerable to health-related risks, which can be the result of, or exacerbated by, environmental conditions in their communities, including air pollution. There are opportunities to pay greater attention to air quality issues affecting Indigenous peoples, support more meaningful engagement and consultation with Indigenous peoples and explore options for greater Indigenous involvement in air quality monitoring and management. This would improve program alignment with the government’s reconciliation priority and the nation-to-nation principle.
• Greater focus on indoor air quality. Canadians spend a large proportion of their time indoors and concentrations of indoor air pollutants can, in some circumstances, greatly exceed outdoor concentrations. For these reasons, increased emphasis on indoor air quality through activities such as additional research and improved information dissemination to P/T, local public health agencies and the general public may be warranted. A changing climate can also affect indoor air quality due to elevated levels of outdoor pollutants, such as smog, wildfire smoke and aeroallergens, that infiltrate into buildings and extreme weather events such as flooding that can lead to the presence of mould indoors.

Although the AAPHI was without a functional horizontal governance structure or approved air quality agenda for most of the period covered by this evaluation, both a formal governance structure and air quality work plan had been approved as of November 2019.

It is unclear to what extent performance information collected and reported under the AAPHI is being used to inform decision making. There are opportunities to revise the logic model and performance measurement framework to ensure that all program activities are adequately represented and to streamline the number of outcomes. There is also an opportunity to ensure that performance reporting occurs consistently over time for all outcomes and for both indoor and outdoor air. Currently, the ultimate outcomes pertain only to outdoor air.

Recommendations

The following recommendations are directed to ECCC’s Associate Assistant Deputy Minister of Environmental Protection Branch, as the lead senior departmental official responsible for the AAPHI, as well as to HC’s Assistant Deputy Minister of Healthy Environments and Consumer Safety Branch and NRC’s Vice President of Engineering. Deliverables below would be submitted to the ADM Oversight Committee of the Addressing Air Pollution Horizontal Initiative for approval.

Recommendation 1: Improve communication and outreach to partners, stakeholders and the general public related to federal air quality activities. This should include:

• finalizing and implementing an air quality communications strategy in collaboration with Health Canada and the National Research Council
• consolidating information related to air pollution and air quality in a central location online and updating this information in a timely manner
• enhancing public awareness of available information on outdoor and indoor air quality, including the Air Quality Health Index
• actively publicizing and disseminating key reports to interested partners and stakeholders, such as local public health authorities, community organizations and health care professionals
• sharing information about the federal government’s air quality agenda with relevant external partners and stakeholders, including the provinces and territories

The evaluation evidence indicates that while the federal departments involved in the AAPHI generate a large quantity of high-quality information about air pollution and air quality, this information may not be effectively reaching the intended target audiences. Improving communications and outreach could help partners, stakeholders and the public to use the information generated by the program to inform and guide their own activities and behaviours.

Recommendation 2: Advance commitments to develop and establish outstanding Base-level Industrial Emissions Requirements, including in regulations to address emissions from petroleum refineries.

The federal commitment to develop and establish BLIERs, including for the refining sector, predates the AAPHI and the period covered by this evaluation. While some progress has been made to establish BLIERs in regulatory and non-regulatory instruments, including the Multi-Sector Air Pollutants Regulations and the regulations to limit VOC emissions from the refining sector, further steps should be taken to advance this work, if the need is reconfirmed.

In the case of refinery emissions, the need for action is underscored by data indicating that Canadian refineries emit far more pollutants than their counterparts in the United States and that one or more of the Canadian Ambient Air Quality Standards (CAAQS) for sulphur dioxide, nitrogen dioxide, ozone and fine particulate matter is being exceeded in seven out of 13 areas where there are petroleum refineries and oil sands upgraders. The Government of Canada indicated that it had begun to develop regulations to address these emissions in its 2018 response to Parliament on the review of the Canadian Environmental Protection Act, 1999.

Recommendation 3: Better integrate air quality and climate change policy and initiatives, while continuing to recognize these as distinct policy areas.

There is considerable support among internal and external participants in the evaluation for better integrating air quality and climate change policy. It is important to continue to recognize these as distinct policy areas. However, a more systematic consideration and accounting of the inter-relationships between them could optimize federal risk management strategies and improve alignment with the federal government’s climate change priority.

Recommendation 4: Support more meaningful engagement and consultation with Indigenous peoples and explore options for greater Indigenous involvement in air quality monitoring and management.

Evidence indicates that Indigenous peoples are particularly vulnerable to health-related risks, which can be the result of, or exacerbated by, environmental conditions in their communities, including air pollution. There is widespread support among evaluation participants for more meaningful engagement with Indigenous peoples on air quality issues to help mitigate these risks. This would also improve program alignment with the government’s reconciliation priority and the nation-to-nation principle.

Recommendation 5: Recognizing that Canadians spend a large proportion of time indoors and that concentrations of indoor air pollutants can be significant, consideration should be given to enhancing the initiative’s focus on indoor air quality.

At present, most AAPHI activities focus on outdoor air quality and there is limited information on the extent to which the public is aware of and using the information generated about indoor air quality. Increased emphasis on indoor air quality is warranted, considering the large proportion of time (90%) that Canadians spend indoors and the fact that concentrations of air pollutants can, in some circumstances, greatly exceed outdoor concentrations. A changing climate can also affect indoor air quality due to elevated levels of outdoor pollutants, such as smog, wildfire smoke and aeroallergens, that infiltrate into buildings and extreme weather events such as flooding that can lead to the presence of mould indoors.

Context

Overview

This report presents the findings of a horizontal evaluation of the Addressing Air Pollution Horizontal Initiative (AAPHI), the cornerstone of the federal government’s efforts to address air pollution. The AAPHI is led by Environment and Climate Change Canada (ECCC) in collaboration with Health Canada (HC) and the National Research Council (NRC). It aims to improve air quality, reduce impacts of air pollution on health and the environment and provide Canadians with the tools to make informed decisions to reduce their exposure to indoor and outdoor air pollutants.

The AAPHI is a continuation of previous work conducted under the Clean Air Regulatory Agenda, which was in place from 2006–07 to 2015–16. The Air Quality Management System (AQMS), a collaborative federal and provincial/territorial (P/T) approach to reducing air pollution, was established in October 2012. The AQMS consists of the following activities:

• Developing, publishing and updating Canadian Ambient Air Quality Standards (CAAQS), which set targets for outdoor air quality and are published as non-binding health and environmental quality objectives under the Canadian Environmental Protection Act, 1999
• Developing, establishing and administering base-level industrial emissions requirements (BLIERs), which set a minimum level of performance for reducing air pollutant emissions for significant industrial sources
• Activities relating to airsheds and air zones, air quality monitoring and reporting and mobile sources
• Reporting to Canadians on the state of the air

Activities to support the ongoing implementation of the AQMS across Canada are an important part of the AAPHI.

AAPHI activities

The AAPHI consists of activities in four main areas: science and reporting; policy and analysis; mitigation actions for outdoor air quality; and indoor air quality activities.

Science and reporting

Science and reporting activities, carried out by ECCC and HC, provide the information required to develop and implement air quality policy and risk management instruments. Foundational research, air quality monitoring and modelling activities identify how air quality is changing and how it is expected to change under baseline conditions and in response to proposed mitigation measures.

Reporting provides Canadians with information on the state of the air in Canada, including information on emission levels and national- and P/T-level long-term trends in air pollution levels. Among other mechanisms, reporting occurs through the interactive State of the Air report, published on the website of the Canadian Council of Ministers of Environment (CCME) as part of the AQMS. In addition, information is provided to raise Canadians’ awareness of and reduce their exposure to, air pollution. This includes daily reporting to Canadians on air quality via the Air Quality Health Index (AQHI).

Policy and analysis

Policy and analysis is carried out by ECCC and HC. This activity consists of providing strategic advice and analysis to support evidence-based policy as well as regulatory and non-regulatory measures intended to improve air quality for Canadians. It encompasses advice and analysis to support the implementation of the AQMS, including activities related to CAAQS, BLIERs coordination, airshed and air zone coordination and management and mobile sources.

This activity also includes managing and supporting Canada’s participation in international/ bilateral transboundary agreements, in particular the Canada-United States (US) Air Quality Agreement and the Convention on Long-range Transboundary Air Pollution under the United Nations Economic Commission for Europe.

Mitigation actions for outdoor air quality

Mitigation actions for outdoor air quality are undertaken by ECCC. This component of the AAPHI consists of the development, establishment and administration of a range of regulatory and non-regulatory instruments to manage emissions from industrial sectors, vehicles, engines and fuels and consumer and commercial products in order to mitigate the human health and environmental risks associated with outdoor air pollution. This activity encompasses BLIERs instruments under the AQMS; non-BLIERs instruments; work related to the Clean Fuels Strategy to enhance alignment with US fuel quality regulations; and enhanced compliance verification for fuels, vehicles and engines.

Indoor air quality activities

Indoor air quality activities are undertaken by HC and NRC. HC’s activities involve collaboration with P/T representatives to identify and promote best practices in regards to improving indoor air quality. This work also includes conducting health risk assessments and research to provide the scientific basis for these actions, as well as communications and outreach activities. HC also leads the National Radon Program as part of the AAPHI, but this work is outside of the scope of this evaluation. NRC works in partnership with government and industry to improve indoor air quality through ongoing technological advancement (e.g., in relation to building materials, ventilation products, technologies for source control and air cleaning) and identification of methodological techniques to assess potential solutions.

The AAPHI encompasses most federal government activities contributing to the overall goal of improving Canadian air quality. Transport Canada is also responsible for developing regulations that affect air quality, but these activities are not part of the AAPHI. Ultimately, the AAPHI expects to achieve clean air for Canadians and reduce the adverse impacts of air pollution on human health and ecosystems. The AAPHI logic model is shown in Appendix A.

AAPHI resources

ECCC, HC and NRC received approximately $90 million per year for AAPHI activities between 2016–17 and 2019–20, as shown in Table 1.

Source: ECCC, HC and NRC financial data; Figures exclude HC’s National Radon Program.

About the evaluation

This evaluation focused on the period from FY 2016 to 2017 to the middle of FY 2019 to 2020 and included all AAPHI activities, with the exception of HC’s work under the National Radon Program, which will be assessed as part of separate evaluation activities. The objectives of the study were to offer insight into the AAPHI’s relevance, efficiency and effectiveness (as per the 2016 Treasury Board Policy on Results), as well as to identify recommendations for possible improvements in the future. Multiple lines of evidence were used, including:

• A review and analysis of publicly available and internal documents and data, including Government of Canada policy-setting documents (e.g., federal Speeches from the Throne), program planning and operational documents, departmental reporting, annual reports, performance measurement data, financial information, internal communications, peer-reviewed and grey literature and other materials.
• In-depth group and individual interviews with 63 key informants, including:
• internal key informants, consisting of representatives from ECCC, HC and the NRC (n=40 individuals)
• external key informants (n=23 individuals), including researchers, representatives of Indigenous organizations and communities, representatives of municipalities, international stakeholders and NRC clients
• Four focus groups involving 29 external stakeholders, including:
• industry (n=10 participants representing the mining, chemicals, steel production, electricity generation and fuels/petroleum refining/oil and gas sectors)
• non-governmental organizations (NGOs) (n=6 participants)
• P/T representatives involved in the AQMS (n=8 participants)
• P/T representatives involved with the AQHI (n=5 participants)

The term “participants in the evaluation” or “evaluation participants” is used in this report to collectively refer to key informants and focus group participants. Except where otherwise noted, reporting on interview and focus group findings focuses on common themes and areas of agreement across and/or within categories of evaluation participant. Appendix B provides a more detailed description of the evaluation approach.

Findings: relevance

Continued need

It is well established that air pollution, which has been concisely defined as “the emission of harmful substances to the atmosphere” (Ritchie & Roser, 2017), has detrimental impacts on the environment, human health and economic prosperity. Factors that influence and contribute to ambient (outdoor) air pollution include meteorology, wind patterns, topography, natural sources (e.g., wildfires), level of industrial activity, transportation and other mobile sources and population density.

Outdoor air pollution can have adverse impacts on lakes, rivers, soil, vegetation and fish and wildlife populations, as a result of processes such as acid deposition and eutrophication. Acid deposition occurs when sulphur dioxide, nitrogen oxide and ammonia emissions react with water, oxygen and/or other chemicals in the air and these newly created chemicals – known as secondary pollutants – are deposited to land and water. Eutrophication occurs when high concentrations of nitrogen and phosphorus stimulate algae blooms.

With regard to human health effects, exposure to even low levels of ambient air pollution is tied to significant adverse human health outcomes, such as asthma and other respiratory problems and premature mortality from strokes, lung cancer and heart disease (WHO, 2018), as well as adverse birth outcomes and mental health disorders (Brunst et al., 2019; Dibben & Clemens, 2015; Khan et al., 2019; Seabrook et al., 2019). Furthermore, the evidence indicates that there is no safe level of exposure for key air pollutants, such as fine particulate matter and ozone. In 2017, air pollution was the fifth leading risk factor for mortality worldwide, responsible for more deaths than malnutrition, alcohol use and physical inactivity (HEI, 2019) and the leading environmental risk factor for mortality in Canada (IHME, 2019).

Indoor air pollution is also linked to adverse health outcomes. According to the United States Environmental Protection Agency (US EPA), indoor concentrations of some pollutants can be two to five times higher than outdoor concentrations (US EPA, 2017). Given that Canadians spend approximately 90% of their time indoors (HC, 2018b), indoor air quality has a significant impact on individual exposures to air pollutants. However, the concentrations of indoor air pollutants and periods of exposure that are needed to produce specific health effects are not well understood, suggesting a need for further research in this area (US EPA, 2014).

Populations most vulnerable to the health effects of air pollution include those with existing respiratory or cardiovascular conditions, seniors, young children and pregnant women (GoC, 2016a), as well as those in close proximity to sources of pollution such as major highways and pollution-emitting facilities. Indigenous peoples can be particularly vulnerable to health-related risks, which can be the result of, or exacerbated by, environmental conditions in their communities, including air pollution (Kovesi, 2012; Weichenthal et al., 2013).

Beyond detrimental impacts on the environment and human health, air pollution generates substantial economic costs by negatively affecting the productivity of sectors such as forestry, agriculture, fishing and tourism and through increased health expenditures and reduced worker productivity. Despite improvements in air quality over time, there remains a significant health and economic burden associated with ambient air pollution. HC has recently estimated that 14,600 premature deaths each year in Canada are attributable to ambient air pollution from human sources and that the health impacts of air pollution total $114 billion per year in socioeconomic costs (HC, 2019a).

Canada has a low level of ambient air pollution compared to some regions of the world and indicators of air pollution and air quality in Canada generally show improvements over time. However, available data also indicate that ongoing efforts are needed. For example:

• Concentrations of the four key air pollutants for which CAAQS have been developed – nitrogen dioxide, sulphur dioxide, fine particulate matter and ozone – decreased between 2002 and 2016 (ECCC, 2018b). However, ambient levels varied by pollutant and by region. More recent data is not available.
• Regional differences in air quality are related to emission sources, geography and meteorology. Transboundary air pollution from the US has a particularly significant impact on air quality within the Windsor-Quebec corridor, while Western Canada continues to experience nitrogen dioxide exceedances mainly from industry and fine particulate matter exceedances mainly from wildfires. Areas of intense industrial activity continue to experience sulphur dioxide exceedances and most of Ontario continues to observe exceedances of ground-level ozone.
• Based on analysis by ECCC, several air zones, in British Columbia, Alberta, Saskatchewan, Manitoba, the Northwest Territories and Ontario, did not achieve the 2015 CAAQS for fine particulate matter, ozone, or both.
• Data for the period from 2015 to 2017 indicate that 25% of Canadians live in areas where measured outdoor concentrations of fine particulate matter, ozone, sulphur dioxide and nitrogen dioxide exceed one or more of the 2020 CAAQS. This proportion is up slightly from 23% between 2014 and 2016 (ECCC, 2020a).
• In its most recently published projections, ECCC reported that although emissions of most air pollutants are expected to decline over the long-term, emissions of volatile organic compounds (VOC), fine particulate matter and ammonia are expected to increase (ECCC, 2018a). ECCC attributed these expected increases to continued growth in the oil and gas sector, a large share of particulate matter emissions originating from open sources and steady growth in animal and crop production emissions and increased use of nitrogen-based fertilizer, respectively.^{Footnote 1}

Ongoing action is also needed to address a variety of new and emerging issues that are likely to have implications for air quality and pollution emissions.

For instance, climate change is expected to exacerbate both the prevalence and negative impacts of air pollution. The increasing temperatures associated with climate change are projected to produce longer wildfire seasons, more severe weather and increased risk of drought. In turn, these are expected to affect both outdoor and indoor air quality. For example, wildfires produce air pollutants such as particulate matter and VOCs, among others. As another example, drought may lead to an increase in fine particulate matter, allergen and dust concentrations in the air in affected areas (Yusa et al., 2015). Indoor air quality may also be affected by climate change. Impacts may occur as result of changing indoor air concentrations as outdoor concentrations change; more frequent breakdowns in building envelopes due to extreme weather events; and reduced ventilation stemming from the implementation of energy-efficiency measures that limit the exchange of indoor air with outdoor air (IOM, 2011).

A related issue is the need to address short-lived climate pollutants, which have warming effects on climate and are also associated with direct adverse human health effects. One example is black carbon, a component of fine particulate air pollution originating primarily from transportation sources, as well as wildfires, solid fuels such as coal and wood and industrial processes. Black carbon emissions are estimated to be the second-strongest contributor to current global warming behind carbon dioxide emissions (Ramanathan & Carmichael, 2008). Since the atmospheric lifetime of black carbon and other short-lived climate pollutants is short (i.e., a few days), their impact on climate and air quality can be efficiently reduced by targeting their primary sources.

Another emerging issue is pollution hotspots, geographic areas whose close proximity to heavy industry or other sources of pollutants results in high exposure. In Canada, examples include the area around Sarnia, known as “Chemical Valley” and the Alberta oilsands. In a 2017 report, the Environmental Commissioner of Ontario drew attention to Aamjiwnaang First Nation near Sarnia. The report noted the community has some of the worst air pollution in the country, as well as higher-than-average frequencies of many illnesses (Environmental Commissioner of Ontario, 2017). A recent study found rates of acute myeloid leukemia significantly higher than the national average clustered in Ontario industrial border cities including Sarnia and Hamilton, potentially implicating exposure to materials/pollution from industry as a risk factor for this disease (Ghawazi et al., 2019).

Other emerging issues include the need to:

• consider multi-pollutant mixtures rather than single pollutants when assessing the potential health and environmental impacts of air pollution (HEI, 2016; US EPA, 2014)
• understand the potential health impacts of microplastics in both indoor and outdoor environments (Cox et al., 2019; Gasperi et al., 2018; Vianello et al., 2019)
• address agricultural sources of emissions (Giannadaki et al., 2018; Jerrett, 2015)

External participants in the evaluation agreed on the continued need for federal involvement in addressing air pollution and air quality, citing the health, economic and other societal costs of air pollution. They also pointed to the need to address new and emerging issues, such as those noted above. Finally, they noted that there are no other programs with similar scope, mandate and capacity as the AAPHI.  

Alignment with federal responsibilities and priorities

AAPHI activities are consistent with the federal government’s roles and responsibilities, as articulated in the Canadian Environmental Protection Act, 1999. The Act provides the authority, among others, to undertake research and monitoring; assess and manage the health and environmental risks posed by toxic substances; regulate vehicle, engine and equipment emissions and fuels; and act in cooperation with other levels of government. ECCC, on behalf of the federal government, has the responsibility to engage with other jurisdictions and countries to address air pollution from sources outside Canada that is crossing borders and impacting Canadian air quality. Furthermore, AAPHI activities are consistent with departmental roles and responsibilities as laid out in foundational legislation, including the Department of the Environment Act, the Department of Health Act and the National Research Council Act and with the federal government’s roles and responsibilities in the AQMS.

All external participants in the evaluation agreed that federal involvement in addressing air pollution and air quality should be maintained and many believe there would be a significant gap in this domain without it. For example, P/T representatives indicated that their jurisdictions are heavily dependent on the federal government’s scientific, technical and financial resources in order to fulfill their roles and responsibilities under the AQMS. These include maintaining the air quality monitoring networks in their regions. They noted that they would not have the capacity to sustain their current level of service to the public without it. Similarly, NRC clients emphasized that NRC has unique expertise and capabilities and that the services it provides fill an important gap.

While supporting an ongoing federal role, both industry and P/T focus group participants emphasized the importance of avoiding duplication with P/T regulatory activities. Related to this point, P/T representatives noted that they have little information about the federal government’s planned next steps and timelines with respect to clean air. They urged the federal government to clearly articulate and share with the P/T its long-term “master plan” or agenda for air quality. This would help them in planning their own activities to avoid duplication and minimize regulatory and reporting impact on industry.

While being supportive of an ongoing federal role, industry focus group participants also noted that as air quality continues to improve, the government will need to have a full understanding of the costs and benefits of additional measures to bring about further improvements, including the possibility of diminishing returns and unintended consequences. They urged ECCC and HC to consult broadly in order to obtain a comprehensive understanding of costs and benefits of any potential mitigation measures under consideration.

Alignment with federal priorities

At the time of the evaluation, relevant federal priorities included environmental protection; economic growth and prosperity; climate change; reconciliation with Indigenous peoples; science, research and innovation; and open and transparent government.^{Footnote 2}  Although the AAPHI generally aligns well with federal priorities, there is considerable support among both internal and external participants in the evaluation for enhancing alignment with the government’s climate change and reconciliation priorities. This is discussed in detail in Section 4.1.

Findings: effectiveness

Activities and accomplishments

During the period covered by this evaluation, progress was made towards delivering planned activities in all of the AAPHI’s core activity areas.

Science and reporting

In the area of science and reporting, ECCC and HC conducted foundational research and maintained monitoring networks and inventories such as the National Air Pollution Surveillance program, the Air Pollutant Emissions Inventory and the National Pollutant Release Inventory. They also performed air quality monitoring in collaboration with the P/Ts and conducted risk assessments on the health and environmental effects of air pollution. For example, comprehensive human health risk assessments for sulphur dioxide and nitrogen dioxide provided the evidence-based health rationale for development of CAAQS for these pollutants.

New research activities initiated during the period covered by this evaluation studied the relationship between exposure to air pollution and different health outcomes, some of which focus on specific demographics and vulnerable populations (e.g., children, the elderly, Indigenous communities). One example is the Sioux Lookout Zone Children’s Environmental Health Study. New monitoring activities were also undertaken in collaboration among ECCC, the Province of Ontario and the University of Toronto to establish a roadside air monitoring network in Toronto to monitor traffic-related air pollution.

ECCC and HC continued to report to stakeholders and the public through numerous means, including the Air Pollutant Emissions Inventory Report; the National Pollutant Release Inventory; the Greenhouse Gas and Air Pollutant Emissions Projections Report; the Black Carbon Inventory Report; the Fleet Average NO_x Emission Performance of Light-duty Vehicles, Light-duty Trucks and Medium-duty Passenger Vehicles Report; and the Health Impacts of Air Pollution report, as well as the State of the Air report.

ECCC and HC continued implementing the AQHI in collaboration with the P/Ts, municipalities, local health regions, public health units and NGOs. As one element of this, ECCC supported several jurisdictions in adapting the AQHI to their regional needs and circumstances. For example, a specific AQHI for sulphur dioxide was developed for Kitimat by the Province of British Columbia, in collaboration with HC, ECCC and the British Columbia Centre for Disease Control. British Columbia has also tailored the AQHI towards fine particulate matter to address smoke during wildfire season. ECCC completed two major updates to the AQHI numerical forecast systems, including one that systematically incorporates pollutant emissions from satellite-detected wildfires and HC provided funding to the Province of Alberta to develop the AQHI mobile app.

HC disseminated information on outdoor air pollution, impacts on health and how to minimize risks to health through outreach and engagement activities and collaboration with external stakeholders. Specific examples of these activities include:

• the Air Health Check campaign in collaboration with Scout Environmental, PharmaChoice and the Running Room
• a Memorandum of Agreement with Nova Scotia Environment to develop and deliver the Healthy Schools campaign
• a Memorandum of Agreement with York municipality to raise awareness of the AQHI among the general population
• the establishment of a new international Community of Practice on wildfire smoke

Policy and analysis

In the area of policy and analysis, ECCC and HC continued to develop Regulatory Impact Analysis Statements, emissions and economic projections, cost-benefit analyses, valuation, competitiveness analyses and other analyses to support the development of regulatory and non-regulatory mitigation measures. Measures that have been proposed and/or implemented are described below under “Mitigation actions for outdoor air quality”.

The departments continued to lead and coordinate the overall implementation of the AQMS, including foundational work in support of the CAAQS development and review process. In collaboration with the P/Ts, new CAAQS for sulphur dioxide and nitrogen dioxide were established and CAAQS for ozone were strengthened. Two guidance documents for air zone management were developed and published and three guidance documents for CAAQS achievement determination (sulphur dioxide, nitrogen dioxide and ozone) were developed. The guidance documents for sulphur dioxide and nitrogen dioxide have been published, with publication of the ozone guidance document pending at the time of writing of this report.

With regard to mobile sources, the Mobile Sources Working Group completed its action plan for FY 2013 to 2014 to FY 2016 to 2017 and as of 2018 was discussing potential new work items for priority sub-sectors such as on-road freight. In 2016, the document Vehicle and Engine Tampering Description and Examples of Acceptable Practices was published on the CCME website.

The AAPHI also initiated the development of a long-term strategy to continue addressing air pollutants and sources that have not yet been addressed.

The program continued its international work, including continued bilateral collaboration with the US under the Canada-US Air Quality Agreement to reduce transboundary air pollution. This work included implementing work plans to address vehicle and engine emissions and emissions from the oil and gas sector and beginning informal discussions on a possible future update to the agreement.

Multilaterally, in 2017, Canada ratified the 1999 Protocol to Abate Acidification, Eutrophication and Ground-level Ozone (the Gothenburg Protocol) and its 2012 amendments and submitted its reduction commitments in 2018. The Protocol addresses transboundary air pollution, in particular air pollutants that contribute to the formation of acid rain and smog and the degradation of water bodies. With the addition of fine particulate matter (including black carbon) to the amended Protocol, it became the first legally binding instrument to also include a focus on short-lived climate pollutants.

Canada participates in The Joint Task Force on the Health Aspects of Air Pollution, established in 1998 within the Convention on Long-range Transboundary Air Pollution. This Task Force evaluates the health evidence for air pollution and develops international air quality guidelines for the World Health Organization (WHO). Canadian scientists participate in a number of other task forces and working groups under this Convention, including the Task Force on Hemispheric Transport of Air Pollution, currently co-chaired by an ECCC scientist and the Working Group on Strategies and Review (the key negotiating body), currently chaired by an ECCC executive.

Canada was also the key proponent for an air pollution-related resolution under the United Nations Environment Programme, which was adopted by the United Nations Environment Assembly in 2017. The resolution calls on member states and the United Nations Environment Programme to undertake various activities to improve air quality, reduce air pollutants and short-lived climate pollutant emissions and work toward a shared global response to air pollution, including by creating a platform for information-sharing and policy cooperation.

Mitigation actions for outdoor air quality

Progress was made in developing, proposing and implementing mitigation actions to reduce emissions of air pollutants from the industrial, transportation, consumer and commercial sectors of the economy, building on previous work under the Clean Air Regulatory Agenda.

The Multi-Sector Air Pollutants Regulations, which were developed under the Clean Air Regulatory Agenda, were put in place. The regulations establish air pollution emission standards set out in a number of BLIERs developed under the AQMS. In particular, the regulations limit the amount of nitrogen oxides emitted from gaseous fuel-fired non-utility boilers, heaters and stationary spark-ignition gaseous fuel-fired engines, as well as the amount of nitrogen oxides and sulphur dioxide emitted from cement kilns.

Under the Core Climate Change Mitigation Program, the Regulations Respecting Reduction in the Release of Methane and Certain Volatile Organic Compounds (Upstream Oil and Gas Sector) and the Reduction of Carbon Dioxide Emissions from Coal-Fired Generation of Electricity Regulations were developed and put in place.These regulations are expected to achieve some of the same air quality objectives anticipated under BLIERs and to result in reductions in air pollutant emissions in addition to greenhouse gas emission reductions.

Non-regulatory instruments, including codes of practice, pollution prevention planning notices and environmental performance agreements, were also developed. These instruments implemented the standards set out as BLIERs for a range of pollutants across several sectors, including iron, steel and ilmenite; aluminum; base metal smelters; iron ore pellets; and pulp and paper.

ECCC amended numerous regulations to manage emissions from the transportation, consumer and commercial products sectors. For the transportation sector, amendments were made to various regulations to limit emissions from on-road and off-road vehicles and engines; and under the Clean Fuels Strategy, amendments were made to various regulations, including the Sulphur in Diesel Fuel Regulations to align Canadian fuel standards with those in the US. ECCC also expanded the quantity and scope of vehicle and engine testing it conducts, established an approach and methodology for enhanced compliance verification for vehicles and engines and implemented enhanced compliance verification for fuels as a five-year enforcement project.

With regard to consumer and commercial products, amendments were made to regulations setting VOC concentration limits for architectural coatings and automotive refinishing products.

In addition, a number of proposed new regulations were published to reduce the release of VOCs from the petroleum sector; to set VOC concentration limits in certain products; and to regulate emissions from off-road large spark-ignition and compression-ignition engines, both mobile and stationary, including those used to produce off-grid diesel-generated electricity in remote locations. A complete overview of mitigation measures for outdoor air implemented and proposed under the Clean Air Regulatory Agenda and the AAPHI is in Table 2.

Table 2: mitigation measures for outdoor air under the Clean Air Regulatory Agenda and the AAPHI

Source: Information provided by ECCC.

Despite these areas of progress, some planned mitigation measures have not moved forward to the extent originally envisioned. Most notably, less progress than planned has been made in developing and establishing some of the BLIERs, including regulations to reduce emissions from the petroleum refining sector.

When federal and provincial and territorial ministers of Environment agreed to move forward with the implementation of the AQMS in October 2012, 14 industrial sectors, three classes of equipment and four pollutants were identified as priorities (see text box). BLIERs were also prioritized by the federal government for implementation based on instrument type (regulatory or non-regulatory), readiness, level of consensus and impact on emission reductions.

To date, as described above, the federal government has codified some BLIERs in the Multi-Sector Air Pollutants Regulations, while many others have been established as non-regulatory instruments. However, the federal government did not move forward as anticipated to propose and establish BLIERs for which consensus was not reached during the process to develop the AQMS. In particular, plans articulated in AAPHI foundational documents to propose emission standards for petroleum refineries have not proceeded, with the exception of regulations to reduce VOC emissions.^{Footnote 5}  In its 2018 response to the House of Commons Standing Committee on Environment and Sustainable Development’s review of the Canadian Environmental Protection Act, 1999 and associated recommendations, ECCC indicated that it has begun to develop regulations to reduce sulphur dioxide, nitrogen oxide and fine particulate matter emissions from refineries (GoC, 2018b). The status of this work is unclear.

External evaluation participants had mixed perspectives with regard to progress on BLIERs to date. On one hand, industry representatives praised the level of consultation and engagement with industry and the government’s “flexible” approach, which they noted resulted in a decision not to proceed with BLIERs in some cases. On the other hand, P/T and NGO representatives characterized advances in this area as insufficient or disappointing and some singled out the limited regulation of petroleum refinery emissions as a weakness of Canada’s approach to addressing air pollution.

The need for action in this area was highlighted in a recent analysis by Ecojustice, based on 2015 emissions data. This analysis showed that most Canadian refineries emit more pollutants than comparable US facilities and that none show a clear trend of reductions since 2005 (Ecojustice, 2018).^{Footnote 6}  Moreover, according to ECCC, one or more of the CAAQs for sulphur dioxide, nitrogen dioxide, ozone and fine particulate matter is being exceeded in seven out of 13 areas where there are petroleum refineries and oil sands upgraders.

ECCC representatives indicated that the program had planned to undertake an exercise to determine whether the pollutants and processes targeted by the BLIERs that never reached consensus are still relevant. This would include an analysis of the existing requirements faced by the affected sectors, the current emissions situation and potential impacts of climate measures. This analysis was to inform decisions about additional air pollutant control measures and to report back to AQMS stakeholders on the federal government’s intentions related to outstanding BLIERs. This analysis has not been completed.

Indoor air quality activities

In response to internal and external stakeholder requests, HC publishes a variety of science-based documents. These serve to inform partners as well as Canadians, on strategies to reduce one’s exposure to indoor air contaminants. During the period covered by this evaluation, HC published Residential Indoor Air Quality Guidelines for acetaldehyde. The guidelines provided information relating to known health risks, sources and recommended exposure limits, as well as ways to reduce exposure. HC also published Indoor Air Reference Levels for 25 VOCs.  In addition, HC contributed to the development and update of several Canadian Standards Association standards with respect to indoor air (i.e., carbon monoxide detectors, formaldehyde, electrostatic air cleaners) and participated as a member of other standard-setting bodies, such as the American Society of Heating, Refrigerating and Air Conditioning Engineers, whose standards are used throughout building codes in Canada. HC also led the development and publication of proposed regulations to limit formaldehyde emissions from composite wood products with departmental partners.

In an effort to educate Canadians and public health officials, HC produced indoor air quality factsheets and infographics for carbon dioxide, mould, asbestos, cooking and air quality, relative humidity, ventilation and maintaining and improving indoor air quality, as well as a document on ventilation on the indoor environment. These were created for a broad audience, but based on rigourous science assessments in order to provide accurate information to Canadians and stakeholders.

HC covered indoor air quality as part of outreach and education activities, including the delivery of Chemical Awareness and Learning Module sessions and use of a Virtual Reality tool; and it delivered Detect to Protect, a public awareness campaign for carbon monoxide, in collaboration with Canadian Tire, Kidde Canada and Scout Environmental. HC also provided support to the British Columbia Centre for Disease Control for the development of a Carbon Monoxide Monitoring and Response Framework in Long-Term Care Facilities (HC, 2018a).

HC collaborated with NRC on a study that tested two interventions to improve indoor air quality in homes with attached garages and collaborated on an NRC-led study involving development of a preventive maintenance protocol for ventilation systems and its testing in 55 homes in Nunavik.

NRC carried out research studies in five areas, including the aforementioned collaborations with HC: improving ventilation, managing formaldehyde, radon control, reducing wildfire smoke indoors and work relating to the Canadian Committee on Indoor Air Quality and Buildings. NRC also engaged in a variety of knowledge translation and dissemination activities to disseminate the research findings to stakeholders.

Access to information and analysis

One of the expected outcomes of the AAPHI is that senior managers and decision makers have access to information and analysis on air quality and air pollution. Based on the document review and interviews with internal key informants, a considerable volume of information and analysis on air quality and air pollution is produced, made available to senior managers within the federal government and used to inform decision-making. Examples include, but are not limited to:

• Air quality modelling analysis supported the development of proposed regulations and the development of CAAQS for sulphur dioxide and nitrogen dioxide.
• Health risk assessments provided the health basis for setting new CAAQS for nitrogen dioxide and sulphur dioxide and the Air Quality Benefits Assessment Tool was used to calculate the benefits of reducing air pollutant emissions for five new or proposed regulations. Additional health risk assessments provided the basis for strengthening the existing ozone CAAQS and will provide the basis for an update to the existing CAAQS for fine particulate matter.
• The Canadian Mercury Science Assessment, published in 2016, supported Canada’s ratification of the Minamata Convention on Mercury.
• ECCC research informed the United Nations Educational, Scientific and Cultural Organization’s review of the Wood Buffalo National Park and subsequently contributed to the ECCC action plan for the park. ECCC research also informed the joint Canada-US State of the Great Lakes 2017 assessment and the first effectiveness evaluation (2017) of the Stockholm Convention on Persistent Organic Pollutants.
• Transboundary pollution analyses were completed in support of the Canada-US Air Quality Agreement, including analysis of trends in North American sulphur and nitrogen dioxide wet deposition and transboundary transport from 1980 to 2015, delivered to the 2017 Canada-US Air Quality Committee meeting.

Information and analysis is also available to a range of stakeholders outside the federal government through the Open Data Portal, the National Air Pollutant Surveillance program and CCME websites and other mechanisms. The evaluation evidence indicates that this information is being accessed and used by some stakeholders to support their decision-making. For example:

• P/T representatives agreed that the ability to access high-quality information and science about air quality generated by the federal government has assisted decision-making in and by their jurisdictions. For example, it allows them to assess long-term air quality trends, better understand the impacts of air quality on human health and the environment, support the development of policies and programs to reduce emissions and inform and educate the public.
• Similarly, industry representatives praised the amount and quality of information and analysis available, although they identified a need for more technical support and guidance for industry to support implementation of and compliance with the Multi-Sector Air Pollutant Regulations and other BLIERs instruments, as well as the CAAQS.

NGOs and other external evaluation participants likewise commented positively on the quality of information and analysis on air pollution and air quality, but also identified some weaknesses with the dissemination of information to stakeholders. In particular, they observed that:

• Key reports, such as the State of the Air report and HC’s study on the health impacts of air pollution, have the potential to inform the development and promotion of policies and programs with respect to healthy built environments, transportation and other areas of public health. However, these reports need to be better publicized and actively disseminated to public health agencies and other organizations at the P/T and local levels.
• Scientific studies and reports are not consolidated in one easily accessible location or portal. Overall, it can be difficult to find information.
• Websites and online reports are not updated in a timely fashion.

A few NGO representatives commented specifically on the State of the Air report, observing that although it holds promise as a platform for disseminating information to Canadians, it does not use the latest data that is publicly available elsewhere on the Government of Canada’s website. They also noted that its presentation of ambient air trends as a national average glosses over geographic trouble spots and pointed out that the report is difficult to use in rural and remote areas of Canada with poor telecommunications infrastructure.

ECCC representatives noted that a National Air Quality Communication Strategy was developed for the department in 2019, with recommendations for enhancing communications to various target audiences, including community organizations, health care professionals, media, regulatees, decision makers and the Canadian public. The Strategy was not approved at the time of the evaluation.

Public awareness and behaviour change

As a result of AAPHI activities, Canadians are expected to have increased awareness of indoor and outdoor air quality, air pollution, related health risks and strategies to reduce exposure and to change their behaviour to reduce their exposure to indoor and outdoor air pollutants.

Outdoor air

There is limited data available on the public’s awareness and use of outdoor air quality information in general and the AQHI in particular. The most recent national survey research, conducted in 2017 (Leger, 2017), found that:

• 34% of Canadians were very or somewhat familiar with the air quality information available in their community
• 1% of Canadians could correctly identify the AQHI without being prompted or assisted, while 27% could recall the AQHI with assistance
• 11% had “moderate to high awareness” of the AQHI, 26% had “lower awareness,” and 63% had “no awareness”, based on a composite index consisting of several questions
• 58% had not taken any specific actions to reduce their exposure to air pollution
• among the minority of Canadians who recalled the AQHI with assistance, about three quarters did nothing differently or could not recall if they did anything differently, while just over one quarter reduced their time spent outdoors

Since the survey was conducted, the AQHI has continued to be implemented in communities across the country. As of 2019, the AQHI was available to 80% of Canadians in 111 communities across all P/Ts (HC, 2019b). An estimated 1,380,000 sensitive individuals were reached by AQHI risk communications (ECCC, 2020a).

NGOs, P/T representatives and other external evaluation participants agreed on the value of the AQHI, pointing to its simple messaging and standardized approach to air quality reporting. However, they suspected (in line with the 2017 survey data) that most Canadians do not know about it and are not accessing it. They identified several potential strategies for increasing awareness of the tool and producing changes in behaviour. These include more extensive collaboration with grassroots organizations and local public health agencies, which they believe are more effective than federal government departments at reaching the public with health information and messaging. They also suggested to put more emphasis on helping individuals understand their own sensitivities; to deliver more messaging on what individuals can do to protect themselves; and to use a more “hands-on” approach to information provision.

In addition, both internal and external participants emphasized the need for ongoing surveillance of awareness and behaviours through regular surveys of Canadians. Program representatives explained that undertaking public opinion research is a priority for the near future.

Indoor air

There is limited information on the extent to which HC’s indoor air quality activities have increased awareness on the part of Canadians or prompted them to implement actions to reduce the related risks.

• Between January 2018 and January 2019, HC’s indoor air quality web pages had just over 188,000 page views. This includes 55,421 views for “Improve indoor air quality”, which provides information on reducing contaminants at the source, improving ventilation and filtering indoor air.
• Of more than 3,000 respondents to an online survey on the Detect to Protect campaign, 69% reported higher awareness levels after visiting the campaign microsite.

HC representatives indicated that a new performance measure to track use of indoor air quality information has been developed and will be reported on in the future. In addition, indoor air will be included in a survey of Canadians planned for FY 2020 to 2021, which will examine the effectiveness of HC’s health messaging in relation to air quality.

All NRC clients who were interviewed agreed that NRC’s research and evaluation activities addressed their needs in various ways and informed their decision-making. For example:

• Results from a study on improved ventilation and indoor air quality for northern and remote communities will be used to support the development of the First Nations Housing Guide.
• NRC’s research on novel radon barriers has been used to develop and issue an evaluation technical guide in collaboration with the Canadian Construction Materials Centre. The guide has been used by three companies for six product evaluations. Three products gained market access following positive evaluations.
• NRC’s research on reducing radon ingress through passive soil depressurization generated datasets that enabled the Canadian General Standards Board Technical Committee to produce two national standards (radon in existing construction and radon in new construction).

In some cases, NRC evaluated solutions and technologies have experienced market uptake. For example, 16 radon-controlling products have been validated for industry and are now commercially available.

Compliance with air pollution measures

Overall, there is limited information on the extent to which target audiences are complying with federal regulatory and non-regulatory air pollution mitigation measures. As already described, ECCC took steps during the period covered by this evaluation to strengthen compliance review for engines, vehicles and fuels. However, relevant performance data will not be reported by the program until 2021.

• According to the most recent available data, reported in ECCC’s Departmental Results Report for FY 2018 to 2019, 6% of regulatees were subject to fulsome review related to compliance with the fuels regulations (ECCC, 2020a). Previous Departmental Results Reports indicated that self-reported compliance with the fuel quality regulations was 100% or close to 100% (ECCC, 2017b, 2018c).
• With regard to vehicles and engines, the Departmental Results Report for FY 2018 to 2019 indicated that ECCC performed 91 testing rounds for vehicles and engines, a 25% increase from the average in FY 2014 to 2015 and FY 2015 to 2016 (ECCC, 2020a).
• As it has since the 2004 model year, ECCC continues to publish the annual Fleet Average NO_x Emission Performance of Light-duty Vehicles, Light-duty Trucks and Medium-duty Passenger Vehicles Report, which outlines the emissions performance of the new vehicle fleet at a company-specific level.

The current performance measurement framework also identifies several performance indicators relating to compliance for the Multi-Sector Air Pollutants Regulations. However, as with the fuel quality regulations, relevant performance data are not yet available, since most of the emissions standards are not yet in force. As such, performance data will be reported in the coming years. On June 1, 2019, ECCC received the first annual report for 2018 monitoring data for regulated cement facilities. These data showed that 100% of the cement facilities had installed the required Continuous Emissions Monitoring Systems to monitor their nitrogen oxide and sulphur dioxide emissions, which will allow reporting in support of meeting their emissions targets in the future (ECCC, 2020a).

Reporting on compliance with non-regulatory instruments is not part of the current performance measurement framework, although these usually include requirements for performance reporting. For example, ECCC publishes regular progress summaries for performance agreements for base metals smelters, aluminum and iron ore pelletizing. A 2017 progress report on the code of practice for the reduction of VOC emissions from cutback and emulsified asphalt indicates that 17% of the products had a VOC content exceeding the code’s recommended practices (ECCC, 2019). In response, ECCC indicated its intent to enhance compliance promotion.

Reduced emissions

The risk mitigation measures for outdoor air pollutants introduced under the Clean Air Regulatory Agenda and the AAPHI are intended to reduce emissions of these pollutants. Available data indicate that emissions of fine particular matter, sulphur dioxide, nitrogen oxides, VOCs and carbon monoxide from sources subject to mitigation measures have declined since 2006. As shown in Table 3, three-year average emissions declined between 2006 to 2008 and 2016 to 2018, with the percentage reduction ranging between approximately 26% and approximately 63%, depending on the pollutant and source. However, for fine particulate matter and carbon monoxide, the percentage change was slightly lower over this period compared to the period between 2006 to 2008 and 2015 to 2017. This indicates that emissions of these pollutants from sources subject to mitigation measures have increased slightly in recent years, despite the existence of these measures.

Source: Data provided by ECCC.

Furthermore, preliminary calculations using 2018 data from the Air Pollutant Emissions Inventory show that Canada’s indicative reduction targets under the Gothenburg Protocol have been attained for VOCs and fine particulate matter and reduced further than the target set for sulphur dioxide.^{Footnote 7}  As of 2018, further reductions of nitrogen oxides were required in order to meet the 2020 target. See Table 4.

Source: (ECCC, 2020b); Note: Canada’s commitment for fine particulate matter excludes the following open sources: emissions from road dust, construction operations and crop production.

It is important to note that although reductions have been achieved, total emissions of VOCs and nitrogen oxides are still significant (in excess of 1,500,000 tonnes each per year) and, moreover, have increased since 2016. Total emissions of fine particulate matter have trended upward since 2009, although there has been a slight decline in total emissions of fine particulate matter when open sources (road dust, construction operations and crop production) are excluded. Total emissions of sulphur dioxide have declined steadily since 2005. Figure 1  depicts emission trends from 2005 to 2018 for these pollutants, based on data from the Air Pollutant Emissions Inventory.

Source: (ECCC, 2020b).

Notes: PM2.5(1)=fine particulate matter emissions from all sources; PM2.5(2)=fine particulate matter emissions excluding emissions from the following open sources: road dust, construction operations and crop production.

Clean air and reduced adverse impacts

The AAPHI has two ultimate outcomes: Canadians have clean air and adverse impacts on human health and ecosystems are reduced.

The first outcome – Canadians have clean air – is difficult to define and measure. Many indicators could potentially be used and are arguably necessary, to provide the complete picture. For outdoor air, the AAPHI measures progress toward this outcome using two indicators related to the CAAQS.^{Footnote 8} 

The first indicator is the percentage of Canadians living in areas where air quality standards are achieved, with a target of 85% of Canadians in 2030.

• During the period from 2014 to 2016, 77% of the Canadian population lived in areas where measured outdoor concentrations were below the 2020 CAAQS for fine particulate matter, ozone, sulphur dioxide and nitrogen dioxide; this compares favourably with 60% in the baseline year of 2007 (ECCC, 2020a).
• More recent analysis indicates that this proportion slipped to 75% over the period from 2015 to 2017 as a result of increases in particulate matter concentrations in several areas in Western Canada due to wildfire episodes, as well as a small increase in ozone in a few communities in Ontario and Quebec.

The second indicator is the percentage of P/T air zones that achieve the CAAQS, with a target of 85% by 2030. P/Ts are responsible for reporting on the status of their air zones through annual air quality reports. While many have published reports with up-to-date information, not all jurisdictions have published data. As a result, this indicator was not calculated at the time of the evaluation.

Table 5 presents summary information, based on ECCC analysis of P/T air quality reports and 2015 to 2017 ambient data available from the National Air Pollutant Surveillance program where jurisdictional reports were not available.^{Footnote 9}  Based on this analysis, 13 out of 42 air zones (30%), in British Columbia, Alberta, Saskatchewan, Manitoba, the Northwest Territories and Ontario, did not achieve the 2015 CAAQS for fine particulate matter, ozone, or both.

Source: Data provided by ECCC.

Overall, although a majority of Canadians “have clean air” based on these indicators, a sizeable minority does not. Furthermore, the evidence indicates that from a health perspective, there is no safe level of exposure for many key air pollutants, including fine particulate matter and ozone.

The second ultimate outcome is reduced adverse impacts on human health and ecosystems. For human health, the performance indicator is the change in the number of deaths per year attributable to air pollution. Most recently, the 2019 edition of the Health Impacts of Air Pollution report estimated 41 per 100,000 Canadians suffered premature deaths per year as a result of air contaminants, specifically particulate matter, nitrogen dioxide and ozone (HC, 2019a). The 2019 estimate is slightly lower than the 2017 estimate of 42 per 100,000 and is due to lower air pollution levels on average, suggesting that some reduction in mortality risk has occurred.

However, in absolute terms, the number of Canadians who die prematurely each year due to air pollution remains significant – 14,600 according to HC’s latest estimates. Furthermore, HC’s analysis shows that air pollution remains responsible for a variety of morbidity outcomes, including 2.7 million asthma symptom days and 35 million acute respiratory symptom days per year (HC, 2019a). Thus, Canadians continue to be exposed to levels of air pollution that pose risks to health. HC concluded that despite improvements in air quality over time, air pollution continues to have impacts on population health in Canada and that ongoing action is needed.

For ecosystems, the following performance indicator has been developed: area of exceedance of ecosystem critical loads of acidity. The performance target is the area of acidity critical load exceedance from a baseline (to be established). Reporting on this indicator will occur in the future. Given its highly technical nature, consideration could be given to developing accompanying explanatory material for the layperson and supplementing this measure with other measures more accessible to the general public.

Findings: efficiency

Program design

In general, the design of the AAPHI is regarded by internal and external evaluation participants as sound. There is also widespread agreement that the strengths of the initiative include:

• The consultative, collaborative approach taken to deliver the AQMS, the AQHI and other air quality activities.^{Footnote 10} 
• The federal government’s research, science and technical expertise and capacity, which is seen as extremely robust and critically important.
• The standardized, consistent approach to air quality monitoring and reporting through the National Air Pollutant Surveillance program, the AQHI and the CAAQS.
• The focus on transparency and reporting to the public through the AQHI, air zone reporting under the AQMS and numerous other reporting mechanisms.

Canada’s active participation and leadership in international collaborations related to air quality research and management is also seen as an important strength. Key informants from all three partner departments reported that as a result of Canada’s deliberate efforts on the international stage:

• Several countries have expressed interest in, adopted, or adapted the AQHI (Hong Kong, Ireland, United Kingdom, China, Mexico, India).
• Work done under the AAPHI has led to the establishment of an international Community of Practice around wildfire smoke and smoke response.
• Countries around the world look to Canada for its expertise in evaluating the health risks of air pollution.
• NRC is consulted internationally for its expertise in evaluation of technological solutions to improve indoor air quality.

All international key informants consider Canada to be a constructive participant in international collaborations and noted that in recent years it has taken on an increasingly visible leadership role. They encouraged Canada to continue playing this leadership role.

Several possible improvements to the design of the AAPHI emerged from the evaluation.

Better integration between air quality and climate change policy

The precursor to the AAPHI, the Clean Air Regulatory Agenda, addressed both air quality and climate change. However, the AAPHI focuses specifically on air quality, while the Pan-Canadian Framework on Clean Growth and Climate Change (henceforth the Pan-Canadian Framework) has been implemented as a distinct initiative to address climate change.

At present, some AAPHI activities contribute to addressing climate change. Examples include development and implementation of industrial emissions requirements to reduce air pollutants (some of which are linked to climate change), as well as measures to reduce air pollutants from on-road and off‑road vehicles, engines and fuels. In addition, the health and environmental co-benefits from reduction of air pollutants associated with regulations targeting greenhouse gas emission reductions are quantified as part of the cost-benefit analyses for Regulatory Impact Analysis Statements for those regulations. HC has identified “climate linkages” as a theme within its air research funding program and both HC and ECCC personnel are engaged in research exploring climate change, air pollution and air quality. In addition, in July 2017, as part of the Core Climate Change Mitigation Program, ECCC published its Strategy on Short-Lived Climate Pollutants as part of a holistic approach for meeting climate and air quality objectives (ECCC, 2017a).

Nevertheless, there is widespread agreement across internal and external evaluation participants that a more integrated approach to addressing climate change and air pollution should be taken, while recognizing that these are and should be maintained as, distinct policy areas. They identified both a need to better understand the impacts of climate change actions on air pollutants and air pollution actions  on greenhouse gases. As well, they noted that work is required to identify ways to minimize negative impacts and maximize co-benefits in order to optimize risk management strategies. Better integrating air quality and climate change would enhance alignment of the AAPHI with the federal government’s climate change priority.

Greater focus on Indigenous peoples

At the time of the evaluation, some AAPHI activities involve engagement with Indigenous peoples and/or focus on issues relating to Indigenous peoples and air quality. Indigenous organizations are identified in program documentation as a key stakeholder and there is Indigenous representation on some governance committees related to the AQMS. In accordance with its Indigenous Peoples Engagement Strategy, the Energy and Transportation Directorate of ECCC engages with Indigenous groups during regulatory development; holds annual bilateral meetings with its Indigenous partners; and supports engagement with Aamjiwnaang First Nation on air pollution initiatives.

In addition, both HC and NRC have undertaken research studies exploring issues related to air pollution and Indigenous populations. For example, NRC studies have focused on improving ventilation and indoor air quality in homes in Canada’s North. As well, HC is collaborating with Indigenous Services Canada and the Children’s Hospital of Eastern Ontario to fund the Sioux Lookout Zone Children’s Environmental Health Study, which is the largest study of indoor air quality on First Nation reserves at the current time.

Nevertheless, many internal and external evaluation participants believe more should be done to address the needs of and engage with Indigenous peoples with regard to air pollution and quality. Several specific suggestions for improvement came from key informants representing Indigenous communities and organizations.

• One suggestion was providing Indigenous communities and organizations with funding to engage scientific and technical expertise, which would enable them to participate more meaningfully in consultations related to air quality.
• Another suggestion was that the Government of Canada pilot an approach similar to that used by the US EPA, under which tribal authorities can have responsibility for air quality management on tribal lands under the Clean Air Act. This includes, among other things, the ability to develop a Tribal Implementation Plan for achieving the US National Ambient Air Quality Standards (the US equivalent of the CAAQS) (US EPA, 2016b), as well as the ability to develop and implement tribal air monitoring programs and emissions inventories (US EPA, 2016a). It was noted that such an approach – a form of environmental self-government – would be consistent with the nation-to-nation principle and the concept of reconciliation.

While acknowledging the need for improvement, program representatives noted that unlike the Pan-Canadian Framework, the AAPHI lacks specific resources dedicated to Indigenous engagement.^{Footnote 11}

Greater emphasis on indoor air quality

The need for more effort on indoor air quality was identified by a subset of external evaluation participants positioned to comment on indoor air. Their suggestions included updating the indoor air quality guidelines; conducting more research relating to VOCs, mould and other indoor contaminants; and enhancing information dissemination to P/Ts, local public health agencies and the general public. They also suggested addressing the relative lack of information to the public about the risks of exposure to consumer products, particularly cleaning products.

Program representatives involved in indoor air quality activities identified several factors that support an increased focus in this area. These include the large proportion of time (90%) that Canadians spend indoors (HC, 2018b), along with the fact that indoor concentrations of some pollutants can, in some circumstances, exceed outdoor concentrations. Internal key informants also noted that environmental events associated with climate change, such as heatwaves and wildfires, can impact indoor air quality.

Governance

The AAPHI did not have a functioning formal governance structure or approved air quality agenda for most of the period covered by this evaluation. ECCC key informants who were positioned to comment on governance reported that although such a structure existed under the Clean Air Regulatory Agenda, it eventually became inactive and efforts to resuscitate it were delayed by high turnover at the senior management level within ECCC. They noted that in its absence, some planned resources for some air quality activities shifted to other departmental and Government priorities, such as climate change and plastics, without being formally reallocated. Reported consequences included delays in implementing the airshed pillar of the AQMS, pending a full examination of priorities within the CCME portfolio. It also led to inability to provide federal support to AQMS working groups, complete timely updates to the State of the Air website and engage in strategic thinking about potential new areas of work. As well, challenges were experienced in meeting internal data demands and coordinating across all parts of the program.

A formal AAPHI governance structure and Air Quality Program Workplan were approved in November 2019. Internal key informants who spoke on the matter of governance believe that a functional governance structure and approved workplan are essential to advancing federal air quality initiatives in an efficient and effective manner.

At the working level, evaluation findings suggest a need to improve understanding of the roles and responsibilities of those involved in the AAPHI and to ensure consistent communication and coordination. Program representatives generally reported having a good understanding of the roles and responsibilities of their own organizations, as well as the roles and responsibilities of the other organizations within ECCC, HC and/or NRC with which they regularly interact. However, some admitted not having a strong grasp of how their organization fits into the Initiative as a whole, or said they do not necessarily understand who all the players are in the AAPHI and what their respective roles and responsibilities are. A suggested area of improvement was more clearly articulating roles and responsibilities and ensuring shared understanding. ECCC representatives reported that work is currently taking place to clarify roles and responsibilities.

Representatives from all three departments agreed on the value of regular internal communications and coordination to ensure mutual understanding of roles and responsibilities, share information, identify areas of mutual interest and potential collaborations and achieve efficiencies. However, there were differing views about the extent to which such communication and coordination is currently taking place. It appears that although some efforts are being made to communicate and coordinate internally, this is not always being done consistently or may not always involve all relevant internal partners.

Resources

Between FY 2016 to 2017 and FY 2018 to 2019, 95% of planned resources were spent, as shown in Table 6. Both ECCC and HC underspent relative to plans. They spent 96% and 91%, respectively, of the resources planned for the initiative over this period. NRC spent slightly more than planned (108%), although the absolute dollar value is small.

Source: ECCC, HC and NRC financial data. Figures exclude resources for HC’s National Radon Program.

Beyond noting that some planned activities were affected by departmental reallocations to other priorities, internal key informants did not provide detailed explanations for these variances. In addition, as discussed in Section 4.2, ECCC key informants noted that some AAPHI resources shifted without being formally reallocated. As a result, available financial data may not reflect the true extent to which ECCC used AAPHI resources for AAPHI activities.

Representatives from all three departments agreed that the resources originally planned for the AAPHI were adequate to support planned activities.

Performance measurement and GBA+

Performance measurement

A logic model and performance measurement framework have been developed for the AAPHI. The AAPHI has been incorporated into ECCC’s and HC’s Performance Information Profiles for Air Quality. Select indicators from the AAPHI performance measurement framework are incorporated into public reporting by ECCC, HC and the NRC through their Departmental Results Reports.

While performance measurement and reporting are occurring, the extent to which performance information is being used internally to inform decision making is unclear. In interviews, most internal key informants focused on describing challenges associated with performance measurement, rather than describing if and how performance information is used. Identified challenges include, but are not limited to:

• difficulty discerning trends, as a result of changing indicators over time;
• choice of indicators being based on data availability; and
• the complexity involved in calculating many performance indicators, which produces time lags in reporting.

From an evaluation perspective, there are opportunities to improve performance measurement and reporting by revisiting the logic model and performance measurement framework. Areas for improvement include:

• Ensuring that all program activities are appropriately represented. For example, public and stakeholder outreach and awareness activities are subsumed within “science and reporting”.
• Streamlining the number of outcomes. The current logic model contains approximately 15 outcomes, some of which are more accurately described as outputs or activities. Consideration could be given to consolidating some of these outcomes.
• Ensuring that performance measurement occurs consistently over time for all expected outcomes and for both outdoor and indoor air. Outcome information is notably lacking in relation to public awareness and behaviour change. Ultimate outcomes, i.e., “clean air” and “reduced adverse impacts”, pertain only to outdoor air.

Examining existing logic models for similar federal environmental and health regulatory programs, such as the Chemicals Management Plan, could help identify core activity areas and outcomes that are similar across these kinds of programs and lead to greater consistency of the AAPHI logic model with those of similar programs at the federal level.

Gender-based analysis plus (GBA+)

GBA+ was conducted during the development of the AAPHI program authorities. It has been incorporated into some AAPHI activities. For example:

• Health risk assessments and Regulatory Impact Analysis Statements routinely include consideration of vulnerable populations.
• The AQHI provides messaging tailored to vulnerable populations.
• Research studies by HC and NRC explore issues related to air pollution and vulnerable populations, such as children with respiratory symptoms and residents of Canada’s north.
• HC has identified performance measures that incorporate vulnerable populations and, in summer 2019, conducted a GBA+ analysis of the percentage of Canadians who live in areas that meet the CAAQS.
• ECCC has begun work on identifying ways to address the poor air quality experienced by some First Nations communities in geographic hotspots, such as communities downwind of large industrial complexes in northern and western Canada and in southwestern Ontario.

Internal key informants from both ECCC and HC noted that the data needed to support GBA+ is not always available.

Conclusions, recommendations and management response

Conclusions

Relevance

The evaluation confirmed an ongoing need for federal action to address air pollution and air quality. Canada has low levels of ambient air pollution compared to some regions of the world and air quality in the country has generally improved over time. However, the available data are not uniformly positive. One quarter of Canadians live in areas where measured outdoor concentrations of fine particulate matter, ozone, sulphur dioxide and nitrogen dioxide exceed one or more of the 2020 CAAQS. Although emissions of most pollutants are expected to decline in Canada over time, some (e.g., emissions of VOCs, fine particulate matter and ammonia), are expected to increase. Moreover, the significant health and economic burden associated with air pollution – estimated at 14,600 premature deaths and $114 billion in socioeconomic costs per year – along with the potential implications for air pollution and air quality of climate change and other emerging issues, underscore the need for ongoing effort.

The AAPHI aligns with federal roles and responsibilities. Furthermore, participants in the evaluation agreed that the federal government plays an important and valuable role in addressing air pollution and air quality and that this role should continue. However, both industry and P/T focus group participants emphasized the importance of avoiding duplication with P/T regulatory activities. P/T representatives urged the federal government to clearly articulate and share with the P/T its long-term “master plan” or agenda for air quality. This would help them in planning their own activities to avoid duplication and minimize regulatory and reporting impact on industry.

Although the AAPHI aligns with current federal priorities, there is considerable support among internal and external participants in the evaluation for enhancing alignment with the government’s climate change and Indigenous reconciliation priorities.

Effectiveness

Key accomplishments of the AAPHI during the period covered by this evaluation include the ongoing implementation of the AQHI across Canada; the development of new CAAQS for sulphur dioxide and nitrogen dioxide and revised CAAQS for ozone; the implementation of the Multi-Sector Air Pollutants Regulations and a number of non-regulatory instruments establishing BLIERs for several pollutants and sectors; and the ratification of the 1999 Protocol to Abate Acidification, Eutrophication and Ground-level Ozone (the Gothenburg Protocol) and its 2012 amendments.

Some planned mitigation measures for outdoor air have not advanced to the extent originally envisioned. These include the establishment, through regulatory or non-regulatory instruments, of BLIERs for a number of sectors where consensus was not reached during the AQMS development process. This includes planned regulations to reduce emissions of pollutants other than VOCs from the petroleum refining sector. Despite the lack of consensus, there was an expectation that the federal government would move forward appropriately to establish BLIERs for these sectors. To date, however, non-consensus BLIERs, including for the refining sector, have not been established or otherwise dealt with.

The need for action on this front is supported by data indicating that Canadian refineries emit far more pollutants than their counterparts in the United States and that one or more of the CAAQS for sulphur dioxide, nitrogen dioxide, ozone and fine particulate matter is being exceeded in seven out of 13 areas where there are petroleum refineries and oil sands upgraders. The Government of Canada indicated that it had begun to develop regulations to address these emissions in its 2018 response to Parliament on the review of the Canadian Environmental Protection Act, 1999.

The evaluation found evidence of progress toward some of the expected results of the AAPHI, although in some cases, evidence is lacking to support firm conclusions.

• Access to information and analysis. A considerable volume of information and analysis on air quality and air pollution is produced and used to inform decision making by a range of internal and external stakeholders. There was general agreement on the high quality of the information and analysis available. Perceived weaknesses include key studies and reports not being well publicized, information not being consolidated in one easily accessible location and websites and reports not being updated in a timely fashion. ECCC developed but has not yet implemented a National Air Quality Communication Strategy, with recommendations for enhancing communication to various target audiences
• Increased public awareness and behaviour change. The AQHI is the federal government’s main mechanism for disseminating information about outdoor air quality and health risks to the Canadian public. External participants in the evaluation praised it for its simple messaging and standardized approach to air quality reporting. However, the most recent national survey, from 2017, indicates that few Canadians are aware of it or are using it to guide their behaviour. With regard to indoor air, HC has limited information on the extent to which the public is aware of and using the information it generates. NRC’s indoor air research and evaluation activities have addressed the needs of its clients in various ways and informed their decision making. Some NRC evaluated solutions and technologies have experienced market uptake.
• Compliance with air pollution measures. Overall, there is limited information on the extent to which regulated communities comply with mitigation measures for air pollution. The program has developed performance indicators for recently introduced mitigation measures and compliance data will be reported in the coming years.
• Reduced emissions. Emissions of air pollutants from sources subject to mitigation measures have mainly declined over the last 15 years. However, some emission indicators show evidence of recent increases.
• Clean air and reduced adverse impacts. While a majority of Canadians live in areas where air quality standards are achieved, 25% do not. Furthermore, 30% of P/T air zones did not meet one or more of the 2015 CAAQS for fine particulate matter, ozone, or both. With regard to adverse health impacts, although mortality risk due to air pollution has declined slightly in recent years, the number of Canadians who die from air pollution each year remains significant and air pollution causes a variety of morbidity outcomes. Thus, Canadians continue to be exposed to levels of air pollution that pose risks to their health. For risks to ecosystems, the program has recently developed an acidification indicator and plans to report on it in the future.

Efficiency

Internal and external participants in the evaluation consider the overall design of the AAPHI to be sound. Perceived strengths include its collaborative approach to delivering key activities; its robust research, science and technical expertise and capacity; its standardized approach to air quality monitoring and reporting; and its focus on transparency and reporting to the public.

Canada’s increased profile in recent years as an international leader with respect to air quality research and management is also seen as an important strength. Several countries have expressed interest in, adopted, or adapted the AQHI. Work done under the AAPHI has led to the establishment of an international Community of Practice around wildfire smoke and smoke response. Countries around the world look to Canada for its expertise in evaluating the health risks of air pollution. NRC is consulted internationally for its expertise in evaluation of technological solutions to improve indoor air quality. International stakeholders praised Canada for its contributions on the international stage and encouraged it to continue playing a leadership role.

Several possible improvements to program design emerged from this evaluation.

• Better integration between air quality and climate change policy.Internal and external participants in the evaluation emphasized the need to more systematically consider and account for the inter-relationships between air quality and climate change, from both science and risk management perspectives, while maintaining these as distinct policy areas. They identified both a need to better understand the impacts of climate change on air quality and increased air pollution on climate change as well as a need to consider potential impacts of climate change actions on air pollutants and actions to address air pollution on greenhouse gases. As well, they noted that work is required to identify ways to minimize negative impacts and maximize co-benefits in order to optimize risk management strategies.
• Greater focus on Indigenous peoples. Evidence indicates that Indigenous peoples are particularly vulnerable to health-related risks, which can be the result of, or exacerbated by, environmental conditions in their communities, including air pollution. There are opportunities to pay greater attention to air quality issues affecting Indigenous peoples, support more meaningful engagement and consultation with Indigenous peoples and explore options for greater Indigenous involvement in air quality monitoring and management. This would improve program alignment with the government’s reconciliation priority and the nation-to-nation principle.
• Greater focus on indoor air quality. Canadians spend a large proportion of their time indoors and concentrations of indoor air pollutants can, in some circumstances, greatly exceed outdoor concentrations. For these reasons, increased emphasis on indoor air quality through activities such as additional research and improved information dissemination to P/T, local public health agencies and the general public may be warranted. A changing climate can also affect indoor air quality due to elevated levels of outdoor pollutants, such as smog, wildfire smoke and aeroallergens, that infiltrate into buildings and extreme weather events such as flooding that can lead to the presence of mould indoors.

The AAPHI was without a functional horizontal governance structure or approved air quality agenda for most of the period covered by this evaluation. Evidence from internal interviews indicates that this resulted in some AAPHI resources shifting to address other departmental and Government priorities. However, a formal governance structure and air quality workplan were approved in November 2019. Evaluation findings suggest a need to improve understanding of the roles and responsibilities of federal partners involved in the AAPHI at the working level. There is also room to ensure more consistent communication and coordination among them in order to share information, identify areas of mutual interest and potential collaborations and achieve efficiencies.

A logic model and performance measurement framework have been developed for the AAPHI. However, it is unclear whether the performance information collected and reported used to inform decision-making. There are opportunities to revise the logic model and performance measurement framework to ensure that all program activities are adequately represented; streamline the number of outcomes; and ensure that performance reporting occurs consistently over time for all outcomes and for both indoor and outdoor air. Currently, the ultimate outcomes for the AAPHI pertain only to outdoor air.

Recommendations and management response

The following recommendations are directed to ECCC’s Associate Assistant Deputy Minister of Environmental Protection Branch, as the lead senior departmental official responsible for the AAPHI, as well as to HC’s Assistant Deputy Minister of Healthy Environments and Consumer Safety Branch and NRC’s Vice President of Engineering. Deliverables below would be submitted to the ADM Oversight Committee of the Addressing Air Pollution Horizontal Initiative for approval.

Recommendation 1

Improve communication and outreach to partners, stakeholders and the general public related to federal air quality activities. This should include:

• finalizing and implementing an air quality communications strategy in collaboration with Health Canada and the National Research Council
• consolidating information related to air pollution and air quality in a central location online and updating this information in a timely manner
• enhancing public awareness of available information on outdoor and indoor air quality, including the Air Quality Health Index
• actively publicizing and disseminating key reports to interested partners and stakeholders, such as local public health authorities, community organizations and health care professionals
• sharing information about the federal government’s air quality agenda with relevant external partners and stakeholders, including the provinces and territories

The evaluation evidence indicates that while the federal departments involved in the AAPHI generate a large quantity of high-quality information about air pollution and air quality, this information may not be effectively reaching the intended target audiences. Improving communications and outreach could help partners, stakeholders and the public use the information generated by the program to inform and guide their own activities and behaviours.

Statement of agreement or disagreement

Agree with the recommendation.

Management response

ECCC’s Air Emissions Priorities Division, in collaboration with Health Canada and the National Research Council, will finalize the AQP Communications Strategy, which will lay out an approach to improving communications and outreach and submit it for approval. The Strategy will include, but may not be limited to, specific components to address the recommendations above:

• A centralized online platform for air quality information, including information about the federal government’s air quality agendawith a schedule for updating this information in a timely manner
• A list server, or similar mechanism, where interested parties (e.g., local public health authorities, community organizations and health care professionals) can register to receive studies and publications related to air quality
• A social media approach to enhance public awareness of available information on outdoor and indoor air quality, including the Air Quality Health Index (AQHI)

Recommendation 2

Advance commitments to develop and establish outstanding Base-level Industrial Emissions Requirements, including in regulations to address emissions from petroleum refineries.

The federal commitment to develop and establish Base-level Industrial Emissions Requirements (BLIERs), including for the refining sector, predates the AAPHI and the period covered by this evaluation. While some progress has been made to establish BLIERs in regulatory and non-regulatory instruments, including the Multi-Sector Air Pollutants Regulations and the regulations to limit volatile organic compounds (VOC) emissions from the refining sector, further steps should be taken to advance this work, if the need is reconfirmed.

In the case of refinery emissions, the need for action is underscored by data indicating that Canadian refineries emit far more pollutants than their counterparts in the United States and that one or more of the Canadian Ambient Air Quality Standards (CAAQS) for sulphur dioxide, nitrogen dioxide, ozone and fine particulate matter is being exceeded in seven out of 13 areas where there are petroleum refineries and oil sands upgraders. The Government of Canada indicated that it had begun to develop regulations to address these emissions in its 2018 response to Parliament on the review of the Canadian Environmental Protection Act, 1999.

Statement of agreement or disagreement

Agree with the recommendation.

Management response

ECCC’s Air Emissions Priorities Division will lead an analysis to determine the ongoing need for the outstanding BLIERs given measures put in place since 2012. For the petroleum refining sector in particular, ECCC will conduct an analysis of gaps in coverage of sulphur dioxide and other air pollutant emissions from refineries, upgraders and other facilities that will persist once VOC and methane regulations, carbon pricing and other measures are taken into account. A report to senior management will be provided.

If warranted by the analysis, ECCC (led by the Energy and Transportation Directorate) will develop a proposed approach to address emissions of sulphur dioxide and other air pollutants from refineries and other facilities in the petroleum sector.

Recommendation 3

Better integrate air quality and climate change policy and initiatives, while continuing to recognize these as distinct policy areas.

There is considerable support among internal and external participants in the evaluation for better integrating air quality and climate change policy. It is important to continue to recognize these as distinct policy areas. However, a more systematic consideration and accounting of the inter-relationships between them could optimize federal risk management strategies and improve alignment with the federal government’s climate change priority.

Statement of agreement or disagreement

Agree with the recommendation.

Management response

Preliminary discussions to better integrate climate change and air quality policy have occurred. The Air Quality Program will continue to advance the issue, taking into consideration that it is broader than air quality.

ECCC’s Air Emissions Priorities Division will work with appropriate groups within ECCC and Health Canada towards more systematic consideration of both the impacts on air pollutant emissions of proposed measures to address climate change and the impacts on greenhouse gas emissions of proposed measures to address air pollution. As a first step, a stock-taking exercise to assess the impact of key measures will be undertaken.

Recommendation 4

Support more meaningful engagement and consultation with Indigenous peoples and explore options for greater Indigenous involvement in air quality monitoring and management.

Evidence indicates that Indigenous peoples are particularly vulnerable to health-related risks, which can be the result of, or exacerbated by, environmental conditions in their communities, including air pollution. There is widespread support among evaluation participants for more meaningful engagement with Indigenous peoples on air quality issues to help mitigate these risks. This would also improve program alignment with the government’s reconciliation priority and the nation-to-nation principle.

Statement of agreement or disagreement

Agree with the recommendation.

Management response

A number of air-related initiatives underway include engagement with Indigenous communities to monitor and manage air quality and this will continue.

ECCC’s Air Emissions Priorities Division, in conjunction with Health Canada’s Safe Environments Directorate, will co-lead the development of a stock-taking exercise to identify current activities and explore options for more meaningful engagement of Indigenous peoples in air quality issues with the goal of identifying areas of greatest interest and developing targeted actions.

Recommendation 5

Recognizing that Canadians spend a large proportion of time indoors and that concentrations of indoor air pollutants can be significant, consideration should be given to enhancing the initiative’s focus on indoor air quality.

At present, most AAPHI activities focus on outdoor air quality and there is limited information on the extent to which the public is aware of and using the information generated about indoor air quality. Increased emphasis on indoor air quality is warranted, considering the large proportion of time (90%) that Canadians spend indoors and the fact that concentrations of air pollutants can, in some circumstances, greatly exceed outdoor concentrations. A changing climate can also affect indoor air quality due to elevated levels of outdoor pollutants, such as smog, wildfire smoke and aeroallergens, that infiltrate into buildings and extreme weather events such as flooding that can lead to the presence of mould indoors.

Statement of agreement or disagreement

Agree with the recommendation.

Management response

A number of indoor air quality initiatives are already underway to improve indoor air, which is vital to the health of Canadians.

Building on this, Health Canada, in collaboration with the National Research Council, will develop a plan that identifies activities to increase emphasis on indoor air quality. This plan may include developing calls for research proposals and exploring new options to leverage technologies (such as portable sensors) to better understand and manage indoor air quality issues.

Appendix A: AAPHI logic model

For the purpose of the evaluation, the above outcomes were collapsed to create thematic outcomes that served as the basis for this evaluation’s assessment of program performance. Table 7 illustrates how the AAPHI’s intended outcomes align with the thematic outcomes.

Appendix B: evaluation strategy

Evaluation issues and questions

The following questions related to the relevance and performance of the AAPHI were examined in the evaluation.

Relevance

Continued need for the program

• Is there a continued need for the program?

Alignment with government priorities

• Is the program aligned with federal government priorities?

Alignment with federal roles and responsibilities

• Is the program consistent with federal roles and responsibilities?

Performance

Program efficiency

• Is the program design appropriate for achieving its intended outcomes?
• To what extent is the governance structure clear, appropriate and efficient for achieving expected results?
• Is the program producing intended outcomes at a reasonable cost? Are there alternative, more efficient ways of delivering program outcomes?
• Are performance data being collected and reported? If so, is this information being used to inform senior management and decision makers?

Effectiveness – Achievement of expected results

• To what extent have intended outcomes been achieved as a result of the program?
• Have there been any unintended (positive or negative) outcomes?

Evaluation approach and methodology

Several data collection methodologies were used to address the evaluation issues and questions. Evidence drawn from these methods informed the findings and conclusions.

Review of documents and data

The document and data review served to develop a thorough understanding of the AAPHI and to contribute as a line of evidence to address all evaluation questions. Examples of the types of materials reviewed included key policy setting documents, departmental reports, previous evaluation and audit reports, program/project files and selected research and grey literature.

Key informant interviews

Key informant interviews were used to solicit informed opinions and observations on the evaluation questions from various stakeholders involved in or familiar with the AAPHI. A total of 33 group and individual interviews, involving 63 key informants, were conducted by telephone between September 4, 2019 and February 19, 2020. Table 8 shows the number of key informants by respondent category.

Focus groups

Four focus groups were conducted by teleconference with external stakeholders across Canada, including industry stakeholders, representatives of NGOs and P/T representatives. Potential participants were identified by ECCC and HC. Table 9 shows the number of participants in each group.

Tailored discussion guides were created for each group and these, along with a backgrounder on the AAPHI, were provided to confirmed participants in advance of their scheduled group.

Limitations and mitigation strategies

The evaluation encountered two main limitations while conducting the evaluation and put in place strategies to mitigate their impact.

Appendix C: references

Brunst, K. J., Ryan, P. H., Altaye, M., Yolton, K., Maloney, T., Beckwith, T., LeMasters, G., & Cecil, K. M. (2019). Myo-inositol mediates the effects of traffic-related air pollution on generalized anxiety symptoms at age 12 years. Environmental Research, 175, 71–78.

CAPP. (2019). Crude Oil Forecast. Canadian Association of Petroleum Producers.

Cox, K. D., Covernton, G. A., Davies, H. L., Dower, J. F., Juanes, F., & Dudas, S. E. (2019). Human Consumption of Microplastics. Environmental Science & Technology, 53(12),

Dibben, C., & Clemens, T. (2015). Place of work and residential exposure to ambient air pollution and birth outcomes in Scotland, using geographically fine pollution climate mapping estimates. Environmental Research, 140, 535–541.

ECCC. (2017a). Strategy on Short-Lived Climate Pollutants—2017. Cat. No.: En4-299/2017E-PDF; ISBN: 978-0-660-08363-6

ECCC. (2017b, November 9). Departmental Results Report 2016 to 2017, supplementary tables, Department of Environment, chapter 4.

ECCC. (2018a). Canada’s Greenhouse Gas and Air Pollutant Emissions Projections 2018.

ECCC. (2018b). Canadian Environmental Sustainability Indicators: Air Quality. Cat. No.: En4-144/57-2018E-PDF; ISBN: 978-0-660-28053-0.

ECCC. (2018c, November 20). Departmental Results Report 2017 to 2018, supplementary tables: Department of Environment, chapter 4.

ECCC. (2019, February 20). Volatile organic compound emissions from cutback and emulsified asphalt: Progress report 2017. Official title: 2017 Progress report on the Code of practice for the reduction of volatile organic compound (VOC) emissions from cutback and emulsified asphalt. Cat. No.: En14-271/2-2018E-PDF; ISBN: 978-0-660-28868-0.

ECCC. (2020a). Departmental Results Report 2018 to 2019, supplementary tables: Department of Environment, chapter 2.

ECCC. (2020b, March 13). Canada’s Air Pollutant Emissions Inventory.

Ecojustice. (2018, April 30). Inadequate Pollution Control in Canadian Refineries: Media Backgrounder.

Environmental Commissioner of Ontario. (2017). Good Choices, Bad Choices. Environmental Rights and Environmental Protection in Ontario (p. 290).

Gasperi, J., Wright, S. L., Dris, R., Collard, F., Mandin, C., Guerrouache, M., Langlois, V., Kelly, F. J., & Tassin, B. (2018). Microplastics in air: Are we breathing it in? Current Opinion in Environmental Science & Health, 1, 1–5.

Ghawazi, F., Le, M., Cyr, J., Netchiporouk, E., Rahme, E., Alakel, A., Zubarev, A., Powell, M., Moreau, L., Roshdy, O., Glassman, S., Sasseville, D., Popradi, G., & Litvinov, I. (2019). Analysis of Acute Myeloid Leukemia Incidence and Geographic Distribution in Canada From 1992 to 2010 Reveals Disease Clusters in Sarnia and Other Industrial US border Cities in Ontario. Cancer, 125(11), 1886–1897.

Giannadaki, D., Giannakis, E., Pozzer, A., & Lelieveld, J. (2018). Estimating health and economic benefits of reductions in air pollution from agriculture. Science of The Total Environment, 622–623, 1304–1316.

GoC. (1985a). Department of the Environment Act, R.S.C., 1985, c. E-10 (Consolidation).

GoC. (1985b). National Research Council Act, R.S.C., 1985, c. N-15 (Consolidation).

GoC. (1996). Department of Health Act, S.C. 1996, c. 8(Consolidation).

GoC. (1999). Canadian Environmental Protection Act, 1999, (S.C. 1999, c. 33) (Consolidation).

GoC. (2012). The Air Quality Management System: Federal, Provincial and Territorial Roles and Responsibilities.

GoC. (2015). Speech from the Throne to Open the First Session Forty Second Parliament of Canada.

GoC. (2016a). Health risks of air pollution.

GoC. (2016b). Budget 2016. Cat. No.: F1-23/3E-PDF; ISSN: 1719-7740.

GoC. (2017). Budget 2017. Cat. No.: F1-23/3E-PDF; ISSN: 1719-7740.

GoC. (2018a). Budget 2018. Cat No.: F1-23/3E-PDF; ISSN: 1719-7740.

GoC. (2018b). Follow-up report to the House of Commons Standing Committee on Environment and Sustainable Development on the Canadian Environmental Protection Act, 1999.

GoC. (2019a). Budget 2019. Cat No.: F1-23/3E-PDF; ISSN: 1719-7740,

GoC. (2019b, December 5). Moving Forward Together. Speech from the Throne to open the First Session of the Forty-Third Parliament of Canada.

GoC. (2020, September 23). A stronger and more resilient Canada: Speech from the Throne to open the Second Session of the Forty-Third Parliament of Canada.

HC. (2018a). Health Canada 2017–2018 Departmental Results Report. Catalogue No H1-9 / 32E-PDF; ISSN 2560-810X.

HC. (2018b). Ventilation and the indoor environment. Cat. No.: H144-54/1-2018E-PDF; ISBN: 978-0-660-25488-3

HC. (2019a). Health impacts of air pollution in Canada: Estimates of morbidity and premature mortality outcomes, 2019 report. Cat. No.: H144-51/2019E-PDF; ISBN: 978-0-660-31165-4

HEI. (2016, March 9). Multipollutant Mixtures.

HEI. (2019). State of Global Air 2019. Special Report.

IHME. (2019). Global Burden of Disease 2017.

IOM. (2011, June). Climate Change, the Indoor Environment and Health. Report Brief.

Jerrett, M. (2015). The death toll from air-pollution sources. Nature, 525(7569), 330–331.

Khan, A., Plana-Ripoll, O., Antonsen, S., Brandt, J., Geels, C., Landecker, H., Sullivan, P., Bocker Pedersen, C., & Rzhetsky, A. (2019). Environmental pollution is associated with increased risk of psychiatric disorders in the US and Denmark. PLoS Biol, 17(8).

Kovesi, T. (2012). Respiratory disease in Canadian First Nations and Inuit children. Paediatrics & Child Health, 17(7), 376–380.

Prime Minister of Canada. (2015a). Minister of Environment and Climate Change Mandate Letter 2015.

Prime Minister of Canada. (2015b, November 12). Minister of Health Mandate Letter 2015.

Prime Minister of Canada. (2019a, December). Minister of Health Mandate Letter 2019.

Prime Minister of Canada. (2019b, December 12). Minister of Environment and Climate Change Mandate Letter 2019.

Ramanathan, V., & Carmichael, G. (2008). Global and regional climate changes due to black carbon. Nature Geoscience, 1(4), 221–227.

Ritchie, H., & Roser, M. (2017). Our World in Data: Air Pollution.

Seabrook, J. A., Smith, A., Clark, A. F., & Gilliland, J. A. (2019). Geospatial analyses of adverse birth outcomes in Southwestern Ontario: Environmental Research, 172, 18–26.

Statistics Canada. (2020, July 8). The Decline in Production and Investment in Canada’s Oil and Gas Sector and its Impact on the Economy.

US EPA. (2014, June 11). Exposure and Health Effects of Mixtures of Air Pollutants.

US EPA. (2016a, February 29). Tribal Air Quality Monitoring

US EPA. (2016b, July 21). Basic Information about Air Quality TIPs [Tribal Implementation Plans].

US EPA. (2017, November 2). Introduction to Indoor Air Quality.

US EPA. (2019). Petroleum Refinery National Case Results.

Vianello, A., Jensen, R. L., Liu, L., & Vollertsen, J. (2019). Simulating human exposure to indoor airborne microplastics using a Breathing Thermal Manikin. Scientific Reports, 9(1), 1–11.

Weichenthal, S., Mallach, G., Kulka, R., Black, A., Wheeler, A., You, H., St-Jean, M., Kwiatkowski, R., & Sharp, D. (2013). A randomized double-blind crossover study of indoor air filtration and acute changes in cardiorespiratory health in a First Nations community. Indoor Air, 23(3), 175–184.

WHO. (2018). How air pollution is destroying our health.

Yusa, A., Berry, P., J.Cheng, J., Ogden, N., Bonsal, B., Stewart, R., & Waldick, R. (2015). Climate Change, Drought and Human Health in Canada. International Journal of Environmental Research and Public Health, 12(7), 8359–8412.