Evaluation of Canada’s Clean Air Regulatory Agenda

PDF - 851 KB

Final Report

Audit and Evaluation Branch December 2015

Report Clearance Steps

Planning phase completed - September 2015
Report sent for management response - October 2015
Management response received - April 2014
Report approved by Environment Canada’s Deputy Minister (DM) - October 15, 2015
Report approved by Health Canada’s DM - December 21, 2015

Acronyms used in the report

AQA

Air Quality Agreement

AQHI

Air Quality Health Index

AQMS

Air Quality Management System

BLIER

Base-level industrial emissions requirement

CAAQs

Canadian Ambient Air Quality Standards

CAPMoN

Canadian Air and Precipitation Monitoring Network

CEPA 1999

Canadian Environmental Protection Act, 1999

CESD

Commissioner of the Environment and Sustainable Development

EITE

Emissions-intensive trade exposed

ESB

Environmental Stewardship Branch

FSDS

Federal Sustainable Development Strategy

IAQ

Indoor Air Quality

kt

Kilotonnes

LRTAP

Long-range Transboundary Air Pollution

MSAPR

Multi-Sector Air Pollutants Regulations

Mt

Megatonnes

NAPS

National Air Pollution Surveillance Program

NGO

Non-Governmental Organization

NO

Nitrogen dioxide

NOx

Nitrogen oxides

NRC

National Research Council of Canada

P/T

Province / Territory

PM

Particulate matter

RIAS

Regulatory Impact Analysis Statements

SLCP

Short-lived Climate Pollutants

SO

Sulphur dioxide

SW

Single Window

UNECE

United Nations Economic Commission for Europe

UNFCCC

United Nations Framework Convention on Climate Change

Acknowledgments

The Evaluation Project Team would like to thank those individuals who contributed to this project, particularly members of the Horizontal Evaluation Steering Committee and all interviewees and survey respondents who provided insights and comments crucial to this evaluation. The Horizontal Evaluation Steering Committee was composed of program and evaluation representatives from Environment Canada, Health Canada and the National Research Council of Canada.

The Evaluation Project Team was led by Susan Wharton, under the direction of William Blois, and included Goss Gilroy Inc., and Katheryne O’Connor.  

Table of Contents

• Executive Summary
• 1.0 Introduction
• 2.0 Background
  • 2.1 Program Overview
  • 2.2 Activities
  • 2.3 Governance and Management
  • 2.4 Resource Allocation
  • 2.5 Expected Results
• 3.0 Evaluation Design
  • 3.1 Purpose and Scope
  • 3.2 Evaluation Approach and Methodology
  • 3.3 Limitations
• 4.0 Findings
  • 4.1 Relevance
  • 4.2 Performance - Efficiency and Economy
  • 4.3 Performance - Effectiveness
• 5.0 Conclusions
• 6.0 Recommendations and Management Response
• Annex A Program Logic Model
• Annex B CARA Evaluation-Outcomes
• Annex C Supplementary Expenditure Tables
• Annex D Summary of Findings

Executive Summary

Background

This report presents the findings of the evaluation of the Clean Air Regulatory Agenda (CARA), carried out between July 2014 and March 2015.

CARA was established in 2006 to provide a coordinated framework to incorporate both regulations and alternative (non-regulatory) instruments and deliver an integrated, nationally consistent approach to the reduction of air pollutants and greenhouse gases (GHGs). In 2011, in an effort to consolidate government action on air pollutants, the scope of CARA was expanded to include activities related to the Air Quality Health Index (AQHI) program, Indoor Air Quality and North American Transboundary Air Issues. CARA is directed by Environment Canada (EC) and has two federal partners, Health Canada (HC) and the National Research Council (NRC). CARA has five broad and highly inter-related activity elements: science; reporting (on national GHG and air pollutant emissions); policy; regulatory development; and indoor air quality.

The evaluation examined the three-year timeframe from April 2011 to June 2014 and also, where appropriate, looked at relevant activity from the rest of 2014-2015. Federal expenditures on the program were approximately $100M per year during the period under study.^Footnote1

Methodologies used in the evaluation included a review of documentation and literature, 47 interviews with 61 internal and external stakeholders, an online survey of members of technical working groups for the development of regulations and other instruments under CARA, and 4 case studies focusing on equivalency agreements; the single window reporting system; the National Radon Program; and new outreach approaches for the Air Quality Health Index (AQHI).

Findings and Conclusions

Relevance

There is a continuing need to address the significant health, environmental and economic impacts of air pollution, climate change and indoor air contaminants.

CARA’s objectives are aligned with federal priorities on minimizing environmental threats from climate change and air pollution and managing ecosystem impacts and public health risks. These priorities have been outlined in federal policy and budgetary announcements and in commitments under the Federal Sustainable Development Strategy.

The work being done under CARA supports the strategic outcomes and responsibilities of partner departments EC and HC and helps meet NRC’s responsibilities regarding the support of industry’s implementation of new technological solutions to improve air quality. The roles and mandates of EC and HC under CARA are aligned with the responsibilities and legislative authority of these departments under the Canadian Environmental Protection Act, 1999, plus several other Acts. CARA also supports Canada’s commitments regarding emissions reductions and reporting requirements under international agreements.

Given the shared jurisdiction for these issues, the provinces and territories have been involved in initiatives to address air pollution and climate change for many years and, using a variety of approaches, are taking action through their own air pollution and GHG reduction strategies. Despite the possibility of overlap between federal and provincial/territorial efforts, mechanisms are either planned or in place to ensure multilateral and bilateral cooperation and minimize or avoid duplication.

Efficiency and Economy

The current design of CARA, with its three participating departments and five inter-related activity elements, provides for an appropriate and flexible framework that delivers the capacity and expertise to carry out the CARA agenda. Processes and tools are available to help select the most appropriate regulatory or non-regulatory instruments for addressing air pollution and GHG emissions, and the work to date has reflected key government policy, including regulatory alignment with the US and the use of a sector-by-sector regulatory approach to GHG emissions. Program stakeholders report that the short-term nature of CARA funding has made it more challenging to handle human resources recruitment and retention, not to mention longer term planning.

Several aspects of CARA, including partnering, expenditure prioritization and the use of technology, contribute to efficient delivery. Despite the complexity of this initiative, the roles and responsibilities are clear and well-understood. Although there is no formal CARA-wide governance mechanism, the current approach, which leverages existing interdepartmental and intergovernmental committees and fora, provides an appropriate degree of oversight and coordination. Effective interactions at the working level also support strong governance. An opportunity has been identified to strengthen information sharing and planning between regulatory development and reporting activities in order to more fully integrate reporting expertise into the design of new regulatory requirements, to ensure that new reporting requirements optimize opportunities for harmonization with existing processes and inventories and to avoid an undue reporting burden. The need to clarify commitments so as to further expand/adapt the single window system was also identified.

Performance information on progress on air pollution and GHG emissions trends is available and publicly reported. Performance measurement plans also exist for some regulations, although the measurement and reporting of the impacts of those regulations remain limited, since many of these plans have yet to come into effect.

Achievement of Intended Outcomes

Significant progress has been made towards achieving many CARA outcomes, although some avenues for improvement have been identified.

• Quality science research has been carried out on a collaborative basis in support of policy analysis, the development of regulatory and non-regulatory instruments and service delivery. This research also informs federal stakeholders working on transboundary air quality issues and supports international negotiations and obligations.
• Reporting mechanisms meet international and domestic reporting requirements, support the development of policy and regulations, and inform Canadians. Data collection through a single window reporting system has been established, and this approach is being expanded to include federal and provincial programs and reduce the reporting burden for industry.
• Coverage of the AQHI has increased during the study period to 69% of the Canadian population in 10 provinces and 1 territory, but achieving the coverage target of 90% will depend on ongoing partnerships, the implementation of new AQHI mapping technologies, and discussions with Quebec on the AQHI or a complementary air quality forecasting service. A number of outreach efforts have been undertaken to raise public awareness of AQHI, and the results of event and partner surveys suggest that some progress has been made; however, we have yet to see results from a national omnibus survey that measures awareness and behavioural change.
• An Air Quality Management System (AQMS), including health and environment-based Canadian Ambient Air Quality Standards (CAAQS) and base-level industrial emissions requirements (BLIERs) for the reduction of air pollutants, has been jointly developed and endorsed by all jurisdictions (with the exception of Quebec, which nonetheless supports the objectives of AQMS). A draft regulation covering a sub-set of BLIERs has been published and is now in the final consultation stage prior to finalization, and other non-regulatory instruments have been published or are currently under development. Independent of the AQMS, a suite of air pollutant regulations has also been drafted and implemented for the transportation sector, with additional regulatory amendments currently under development.
• Canada is working on an ongoing basis with the US to reduce transboundary air pollutants. Since 1990, we have seen a reduction in the transboundary movement of air pollutants that cause acid rain and smog. Ambient concentrations of fine particulate matter (PM_2.5) have decreased significantly along the Canada-US border. Discussions are continuing on how to best address PM_2.5 within the framework of the Canada-US Air Quality Agreement (AQA).
• Regulatory development work has been done in collaboration with provincial/ territorial, industry and other stakeholders, and individuals serving on working groups have frequently identified the consultation process as a key contributor to federal efforts to address air pollutant and GHG emissions. Nonetheless, some opportunities to improve the consultation process have been identified, including lengthening the time available to respond to analysis/information, expanding the engagement of non-government organizations and more fully articulating the evidence-base of regulations.
• Participation in compliance promotion/information sessions and regulatory development consultations, plus clearly defined technical requirements, have helped regulatees achieve a high level of understanding of emissions and reporting requirements.
• GHG regulations have been published and are now in effect for the transportation and coal-fired electricity generation sectors-two of the largest emitting sectors. Canada’s total GHG emissions have decreased slightly (3%) since 2005; current trends may still reflect post-2008 economic adjustments. The emissions intensity for the entire economy (ratio of GHG emissions to GDP) has decreased significantly year over year since 2005 but appears now to have stabilized. The Commissioner on the Environment and Sustainable Development (CESD) has concluded that the federal regulatory approach is unlikely to lead to GHG emission reductions sufficient to achieve Canada’s Copenhagen target, given that several regulations have yet to come into force and GHG regulatory development remains ongoing for key emitting sectors (oil and gas, emissions intensive trade exposed and natural gas-fired electricity sectors). A variety of factors, including evolving federal government regulatory priorities, has impacted achievements in this area. EC has pledged to take action on a CESD recommendation that the federal government ramp up its planning and communications around federal measures aimed at achieving GHG reductions targets.^Footnote2
• Equivalency agreements are an emerging mechanism for avoiding federal-provincial/territorial regulatory duplication, stipulating that federal GHG or air pollutant regulations can be stood down should a province or territory have an enforceable regulation providing for an equivalent or better environmental outcome. Such agreements are likely to become an increasingly important tool for minimizing duplication while achieving equivalent environmental outcomes. The communication of a clear framework and guidelines for the use of these complex agreements will be important moving forward.
• There have been improvements in access to information and Canadian awareness of indoor air quality issues. For radon, there is evidence of increased home radon testing and improved construction practices as a result of changes to codes and guidelines governing radon

Recommendations

The following recommendations are based on the findings and conclusions of the evaluation. As the senior departmental official responsible for CARA, the Assistant Deputy Minister (ADM) of the Environmental Stewardship Branch (ESB), working with CARA partners from other branches within EC and HC, will be sent the recommendations as appropriate.

Recommendation #1: Strengthen and formalize linkages between the regulatory and reporting elements of CARA to better harmonize requirements for new and existing regulatory reporting and inventories and improve resource planning for future enhancements.

Recommendation #2: Develop and communicate a policy framework for equivalency agreements in order to facilitate clarity and consistency in their use, better understand future resourcing implications and clarify expectations for provinces and territories.

Recommendation #3: Develop and share best practices / tools for stakeholder consultation to increase the efficiency and effectiveness of the regulatory development consultation process.

Management Response

The ADM of the ESB agrees with the recommendations and has developed a management response that appropriately addresses each of the recommendations.

The full management response can be found in Section 6 of the report.

1.0 Introduction

This report presents the results of the Evaluation of Canada’s Clean Air Regulatory Agenda (CARA), which was conducted by Goss Gilroy Inc. for the Audit and Evaluation Branch of Environment Canada (EC). The evaluation was identified in the Deputy Minister-approved 2013 Risk-Based Audit and Evaluation Plan and carried out in cooperation with Health Canada (HC) and the National Research Council of Canada (NRC) in order to assess the relevance and performance (including effectiveness, efficiency and economy) of CARA. The evaluation was undertaken in fulfilment of a funding commitment in order to support future program funding decisions and respond to the Financial Administration Act and the Treasury Board Policy on Evaluation, which require that an evaluation be conducted at least once every five years.

The document is organized as follows: Section 2.0 provides background information on the EDF. Section 3.0 presents the evaluation design, including the purpose and scope of the evaluation, as well as the approach and methods used to conduct the evaluation. Section 4.0 and 5.0 lay out, respectively, the evaluation’s findings and conclusions. The recommendations and management response are presented in Section 6.0.

2.0 Background

2.1 Program Overview

Purpose and Goal of the EDF Program

Established in 2006, CARA is the centerpiece of the Government’s broader initiative known as the Clean Air Agenda (CAA)^Footnote3, which is aimed at addressing climate change and air pollution and maintaining a clean and healthy environment for all Canadians.

The key focus of CARA is to provide a coordinated framework that provides for the use of both regulations and alternative (non-regulatory) instruments to deliver an integrated, nationally consistent, approach to reducing domestic air pollutant and GHG emissions. CARA activities are grounded in science and economic analysis and developed through extensive consultation with the provinces/territories and stakeholders. Emissions monitoring and reporting are also integral to CARA.

The 2011-2016 renewal of CARA builds on previous initiatives and has a greater emphasis on transport regulations and sector-specific regulations (as opposed to cross-cutting regulations applied throughout the economy) that are aligned with the US as appropriate. Another key focus for this time-frame was the development of a nationally consistent approach to air pollution and the national endorsement of the Air Quality Management System (AQMS), which provides for the establishment of new national ambient air quality standards for key pollutants and new industrial emissions requirements. In 2011, activities relating to the Air Quality Health Index (AQHI) program, Indoor Air Quality (including the Radon Strategy) and North American Transboundary Air Issues, previously included under other CAA themes, were also brought within the ambit of CARA activities as part of an effort to consolidate government action on air pollutants.

2.2 Activities

CARA is led by EC and delivered in partnership with Health Canada (HC) and the National Research Council of Canada (NRC). CARA contains five broad and highly inter-related activity elements:

Science activities under CARA are pursued by EC and HC to support the development of regulatory and non-regulatory measures for reducing GHG and air pollutant emissions and their impact on human health and the environment. Scientific activities in support of CARA fall under the following four areas:

• Atmospheric Research, Monitoring and Modelling supports air quality management in Canada and the evaluation of the impact of air pollution and GHG policies and regulations on human health and ecosystems at national and regional scales. It supports the development and implementation of the main elements of the AQMS, including the empirical basis for Canadian Ambient Air Quality Standards (CAAQS) and base-level industrial emissions requirements (BLIERs), other domestic obligations, service delivery and transboundary issues, plus support for Canadian positions and obligations related to international agreements. Air quality management in Canada relies on data from the air quality monitoring program, which manages sites in urban, rural and remote (regionally represented) areas in Canada.
• Health and Environmental Impacts of Air Pollutants analysis is aimed at gaining an understanding of the nature and severity of the health and environmental impacts of air emissions.
• Science Integration, Accountability and Benefits of Action activities synthesize and integrate the information compiled through the science activities of the Atmospheric Research, Monitoring and Modeling and Health and Environmental Impacts of Air Pollutants, leading to atmospheric science advice and assessments and economic and health benefits analyses.
• Oil Sands Science activities build on existing actions to determine the movement and ultimate destination of air pollutant emissions from the oil sands region and to help examine the impact of industrial activity in the oil sands regions on sensitive downwind ecosystems.

Reporting activities pursued by EC and HC focus on national GHG and air pollutant emissions in support of policy and regulatory development, binding domestic and international reporting commitments (e.g., the United Nations Framework Convention on Climate Change (UNFCCC) and the United Nations Economic Commission for Europe’s (UNECE) Convention on Long-range Transboundary Air Pollution (LRTAP)), and the implementation of the Air Quality Health Index (AQHI). Reporting activities fall under the following three areas:

• Data Collection and Reporting on GHGs provides for the compilation of the GHG National Inventory and the Facility Greenhouse Gas Emissions Reporting Programs, which support the assessment of progress towards reducing emissions.
• Data Collection and Reporting on Atmospheric Pollutants provides for the compilation of the National Air Pollutant Emission Inventory (APEI) and the Black Carbon inventory. These programs underpin the scientific assessment of the effectiveness of air emission regulations. These activities also include delivering on the AQHI, which provides hourly and forecasted index readings and health messages to help Canadians make decisions on minimizing exposure to air pollution and the associated health risks.
• Cross-cutting Data Collection and Reporting includes EC’s Single Window (SW) system, which was launched in March 2010 to ease the reporting burden on industry and facilitate the secure collection, storage and exchange of information. The SW is used by various federal and provincial reporting programs and partners to collect data on GHG emissions and air pollutants.

The Policy activities undertaken by EC and HC include the economic and strategic policy analysis needed to support the development of policy frameworks and regulations in line with Government priorities and existing commitments. Policy activities include:

• Greenhouse Gas Policy provides for strategic analysis, advice and the coordination of domestic climate change policy, along with the technical analysis required to develop a regulatory framework for industrial emissions and support the government’s climate change and clean energy policies. As regards GHGs, this entails maintaining relationships both with other government departments and with the provinces and territories, which includes negotiating and implementing regulatory equivalency agreements^Footnote4 and completing analyses to address emerging issues related to climate change.
• Atmospheric Air Pollutant Policy supports the development and implementation of the main elements of the AQMS, including the development of regulations and other non-regulatory instruments for industrial sector sources of air pollutants and the drafting of ambient air quality standards. Work on air pollutant policy also supports efforts to address transboundary air pollution, notably by managing Canada’s participation in the Canada-US Air Quality Agreement (AQA) and the related work being done under the LRTAP.
• Cross-cutting Analysis comprises macroeconomic analysis of climate change and air pollutant policy impacts, target-related economic analysis and advice, and the selection, competitiveness and distributional analysis, and refinement of particular compliance mechanisms.

Regulatory Development Activities pursued at EC include the development, implementation, administration, compliance promotion and enforcement of regulations and/or alternative instruments (e.g., standards, codes of practice) for the reduction of GHGs and air pollutants. As regards GHGs, Canada is taking a sector-by-sector regulatory approach to tailor its regulations to specific environmental and economic considerations. Regarding air pollutants, Canada is employing both regulations and non-regulatory instruments to reduce emissions from various sectors and equipment types. Where appropriate, GHG and air pollutant regulations are to be aligned with US regulations. CARA regulatory work is structured around five categories of activity aimed at reducing GHG emissions and air pollution from different sectors:

• Electricity Regulations activities include the development and implementation of regulations aimed at applying a stringent performance standard to new coal-fired electricity generation units and old units that have reached the end of their economic life. This also provides for the development of regulations to limit GHG emissions from natural gas-fired electricity generation and reduce air pollutant emissions from the electricity sector.
• Transportation Regulations activities include the development, implementation and administration of regulations to reduce GHG emissions from passenger cars and light trucks and heavy-duty vehicles and engines, as well as regulations for reducing air pollutant emissions from on-road and off-road vehicles and engines and regulations governing the sulphur content in fuel, in alignment with the US.
• Emissions-Intensive Trade Exposed (EITE) Regulations activities include GHG regulatory development covering chemicals and fertilizers production, mining and processing and forest product sectors, as well as air pollutant regulations for industrial boilers and heaters, cement manufacturing and stationary engines, and non-regulatory air pollutant instruments (codes of practice, pollution prevention planning notices and environmental performance agreements) aimed at reducing air pollutants from the pulp and paper, iron and steel, aluminum, base metal smelting, potash and iron ore pelletizing sectors, plus selected equipment groups affecting EITE and other sectors.
• Oil and Gas Regulations activities include the development of regulations to limit GHG emissions from the following activities: upstream oil production, natural gas production and processing, oil sands production and upgrading, and petroleum refineries, as well as regulations for addressing air pollutant emissions from the upstream oil and gas, oil sands, pipelines and petroleum refining sectors.
• Consumer and Commercial Products Regulations activities focus on the Canadian Environmental Protection Act, 1999 (CEPA 1999) regulations and address releases from consumer and commercial products in the auto refinishing and architectural coatings sectors. This also includes regulatory measures designed to reduce emissions of smog-forming volatile organic compounds (VOCs) from consumer and commercial products by extending the coverage of control strategies to sectors not previously addressed.

This regulations activity, besides covering activities specifically related to the development of regulations and alternative instruments, also extends to the following two areas:

• Compliance Promotion and Enforcement includes the coordination, planning and delivery of compliance promotion and enforcement activities that are informed by scientific expertise, testing and analysis for existing and projected CARA priority instruments; the promotion of awareness, understanding and compliance of CARA instruments by regulatees; the provision of scientific and technical support for the development and implementation of CARA instruments; and tracking and reporting on compliance promotion activities.
• Analysis in Support of Regulations includes economic analysis and research in support of regulations, including the preparation of Regulatory Impact Analysis Statements (RIAS), the provision of technical expertise and process support for environmental assessments in support of CARA implementation, the assessment of competitiveness implications, and the provision of legal services in support of regulatory development.

Indoor Air Quality (IAQ) Activities delivered by HC and NRC are intended to improve indoor air quality in Canada through the development of guidelines, codes of practice, mitigation measures, product standards and communication initiatives. The indoor air quality element includes:

• National Radon Program activities inform Canadians about lung cancer risks associated with radon exposure and encourage actions to reduce risks. The work involves assessing radon measurement and reduction methods and solutions, promoting actions that can be taken to reduce radon exposure (such as the adoption of revisions to building codes), and the development of radon measurement, mitigation guidance and communications products for the public and key stakeholder groups (including other levels of government, non-governmental organizations (NGOs), health professionals and the building and facility design industries). The Program also publicizes solutions that Canadians can use to identify and reduce exposure levels in their homes.
• Biological and Chemical Contaminants activities focus on priority air pollutants in residential environments, identifying specific sources of air pollutants, conducting risk assessments, evaluating the effectiveness of different interventions, and communicating information to industry, public health professionals and the public.
• Strategies and Solutions Capacity activities assess indoor air pollutants from a combined technology and health perspective. This work offers guidance for facilitating marketplace acceptance and protocols for reporting emissions from building materials and products, and develops databases for identifying key contaminant sources and exposure routes. This work also contributes to the development of health-based building and maintenance products (including protocols for assessing technologies used to reduce concentrations of indoor air pollutants) and the drafting of indoor air quality guidelines and best practices.

2.3 Governance and Management

CARA is a cross-cutting initiative whose activities span three departments, multiple branches and numerous sub-programs.^Footnote5 There is no single overarching CARA governance mechanism. All three participating departments share responsibility for implementing individual CARA initiatives and for monitoring and reporting on progress on a regular basis. Additionally, there are governance and coordination mechanisms in place for many of the individual components of CARA (e.g., working groups or management committees for air quality monitoring, air quality modelling, the AQHI, SW, CARA research, AQA and regulatory planning).

As the CARA lead, the Environmental Stewardship Branch (ESB) of Environment Canada (EC) is responsible for communications and for the coordinated policy and management processes related to CARA. ESB is also responsible for monitoring and reporting on the results and performance of the overall agenda and the integration of CARA reporting into the Federal Sustainable Development Strategy (FSDS).

2.4 Resource Allocation

Table 1 presents the approximate distribution of CARA expenditures by activity element and department for 2011-2012 through 2013-14. The distribution of expenditures by Branch within each participating department can be found in Appendix C. Appendix C also includes the distribution of expenditures by category (e.g., salary, O&M, G&C).

Table 1: CARA Expenditures by Activity Element & Department- 2011-2014 (000's)

CARA Activity Element - Science

CARA Activity Element - Reporting

CARA Activity Element - Policy

CARA Activity Element - Regulatory development

CARA Activity Element - Indoor air quality

CARA Activity Element - Grand Totals

Sources: Numbers represent the actual expenditures of the CARA Clean Air Agenda based on departmental performance reporting. Numbers may not add up due to rounding.

Table 2 presents the G&C expenditures for CARA from 2011-12 to 2013-14. These expenditures are administered by three EC branches (Meteorological Service of Canada (MSC), Science & Technology Branch (STB) and ESB) under the authority of EC’s Umbrella Contribution Terms and Conditions. For the purpose of this evaluation, this spending has been considered broadly with respect to program efficiency and the achievement of outcomes. G&C expenditures were used primarily by MSC to support outreach programs aimed at increasing awareness of the AQHI; by STB to access unique scientific expertise and capacity within universities in support of CARA priorities; and by ESB to cover UNFCCC dues. These G&Cs are also being reviewed as part of an ongoing evaluation of EC’s Umbrella Terms and Conditions.

Table 2: CARA G&C by EC branch, 2011-2014 (000’s)

Source: Data from EC’s financial system as provided by EC Finance Branch. Numbers may not add up due to rounding.

2.5 Expected Results

For the purpose of the evaluation, program performance was assessed against eight theme-based outcome statements (presented in Annex B), which incorporate the immediate and intermediate outcomes from the CARA logic model (see Annex A). The outcome statements were developed to avoid duplication of related concepts and to align more closely with CARA’s five activity elements.

As far as possible, the evaluation also assesses CARA’s contribution towards achieving the program’s final outcomes:

• Reduced emissions of GHGs and air pollutants from regulated sectors while maintaining competiveness in these sectors;
• Reduced ambient concentrations of PM_2.5 and ground level ozone;
• Canadians change behaviour to reduce exposure to air pollutants to improve their health.

3.0 Evaluation Design

3.1 Purpose and Scope

The purpose of this evaluation is to assess the relevance and performance (including effectiveness, efficiency and economy) of CARA. The evaluation covers the three-year timeframe from 2011-12 to Q1 2014-15; however, relevant activity during the remainder of 2014-15 is also included, as appropriate. The evaluation addresses the activities conducted by EC, HC and NRC across each of the five broad CARA activity elements.

The evaluation of the EDF is part of EC’s 2012 Risk-Based Evaluation Plan, approved by the Deputy Minister, and is intended to support evidence-based decision making in policy, expenditure management and program improvements. The evaluation was conducted in order to meet the coverage requirements of the Treasury Board Policy on Evaluation, which require that all direct program spending be evaluated at least once every five years.

3.2 Evaluation Approach and Methodology

The findings presented in this document are based on four data collection methodologies. Evidence derived from these methods informed the findings and conclusions of the evaluation. ^Footnote6

• Document and literature review: A review of documents was conducted that included descriptive program information, scientific products, departmental and Government of Canada publications and other internal strategic, operational planning and evaluation documents. In addition, a limited review of the literature was also carried out.
• Key informant interviews: A total of 47 individual or group interviews was conducted with 61 individuals between August 2014 and January 2015. The distribution of interviews by respondent category is shown below:
  • Senior management (4 interviews, n=4);
  • Program managers (27 interviews, n=40); and
  • External stakeholders (e.g., provincial and municipal government, industry, non-governmental organizations and international stakeholders) (16 interviews, n=17).
• On-line survey of external working group participants: An on-line survey of non-federal government members of CARA regulatory, standards and alternative instrument working groups was undertaken in January to February 2015. Participants represented private sector companies, industry associations, provincial/territorial and municipal governments, environmental non-government organizations and health organizations. The survey gathered members’ perspectives on the efficiency and effectiveness of stakeholder consultations. In total, 1,147 individuals^Footnote7 from 57 working groups were contacted. Of the individuals with valid contact information (945), 220 completed the survey, resulting in a response rate of 23%. Several of the 220 respondents completed the survey for more than one working group, which meant that 328 detailed working group responses were received.
• Case studies: Four case studies were conducted to provide additional insight into select CARA initiatives, namely:
  • the use of equivalency agreements;
  • single window reporting;
  • new outreach approaches for AQHI; and
  • the National Radon Program.

Case study methodologies included a document review and key informant interviews (n=3-5 for each case study). The case study interviews were in addition to the interviews described above.

3.3 Limitations

The challenges encountered during the evaluation, the related constraints, and the strategies used to mitigate their impact are outlined below.

• Complexity of the initiative: CARA is a large and complex initiative that involves multiple federal departments and five activity elements encompassing both air pollution and GHG emissions. The breadth of CARA meant that it was not possible to obtain detailed feedback on all aspects of the initiative. The evaluation dealt with this limitation by ensuring that data collection addressed the key activities within each of the five activity elements. Case studies were also used to gather more detailed information on selected activities.
• Assessing results within an evolving regulatory agenda: While a number of regulatory commitments were outlined in the original CARA funding request, in some instances the priorities of government and the work taking place under CARA have evolved. Rather than assess achievements solely against final milestones (e.g., regulations published), the evaluation also considered the policy and regulatory work undertaken in response to government’s evolving agenda, as well as the need to respond to emerging issues.
• Attribution of progress on longer-term outcomes: As an area of shared jurisdiction, provincial / territorial (P/T) and other efforts also contribute to emissions reductions; consequently, this study is limited in that it is unable to isolate the impact of federal efforts under CARA. Findings relating to such longer-term outcomes are presented with this in mind.

4.0 Findings

This section presents the findings of the evaluation according to evaluation issue (relevance and performance) and related evaluation question. For each evaluation question, a rating is provided based on a judgment of the evaluation findings. The rating statements and their significance are outlined below in Table 3. A summary of the ratings for evaluation issues and questions is presented in Annex D.

Definitions of Standard Rating Statements:

Acceptable

The program has demonstrated that it has met the expectations with respect to the issue area.

Opportunity for improvement

The program has demonstrated that it has made adequate progress to meet the expectations with respect to the issue area, but continued improvement can still be made.

Attention required

The program has not demonstrated that it has made adequate progress to meet the expectations with respect to the issue area and attention is needed on a priority basis.

Not applicable

There is no expectation that the program would have addressed the evaluation issue.

Unable to assess

Insufficient evidence is available to support a rating.

4.1 Relevance

4.1.1 Continued Need for Program

There is a demonstrated need for initiatives to address the significant health, environmental and economic impacts on Canada of air pollution, climate change and indoor air contaminants.

Air Pollution

• Poor air quality, especially exposure to fine particulate matter (PM_2.5), is associated with a variety of adverse health outcomes;^Footnote8 is linked to about 21,000 premature deaths in Canada annually;^Footnote9 has been deemed to be carcinogenic;^Footnote10 and can have particularly negative consequences for vulnerable groups such as children, the elderly and those with pre-existing medical conditions.^Footnote11
• The Canadian Medical Association has estimated that air pollution costs the Canadian economy billions of dollars each year owing to premature deaths, hospital admissions, emergency room visits, and absenteeism^Footnote12. Non-health related socio-economic impacts (e.g., decline in crop and tree output from farm and forestry operations, costs associated with material breakdown) are also significant^Footnote13
• In addition to health and economic impacts, air pollution contributes to adverse environmental effects, including ecosystem degradation through acid deposition, impacts on wildlife habitat and food from pollutants such as mercury, and effects on vegetation of air pollutants such as ground-level ozone and sulphur dioxide (SO₂).^Footnote14

Climate change

• Climate change is related to the release of GHGs, which warm the atmosphere and oceans and thus affect the natural functioning of the ecosystem, as well as weather conditions, biodiversity, hydrological systems and coastal infrastructure.^Footnote15
• The World Health Organization links climate change to increased mortality and adverse health effects owing to extreme weather events, such as heat-waves and floods, the degradation of water supplies and sanitation systems, and impacts on agriculture.^Footnote16
• Although difficult to quantify, changes to Canada's climate also have economic impacts. Examples include effects on commercial fisheries, forestry, infrastructure and transportation resulting from temperature and precipitation changes, winds and storms, and impacts on shipping, recreational activities and drinking water caused by changes in water levels.^Footnote17

Indoor air contaminants

• Indoor air contaminants can also pose health risks. For example, exposure to radon is the second leading cause of lung cancer for Canadians after smoking. ^Footnote18
• Asthma is one of the greatest chronic health burdens in Canada, affecting approximately 3 million Canadians and 13% of children. Exposure to indoor pollutants such as formaldehyde, nitrogen dioxides (NO₂), VOCs and allergens are known to aggravate this condition.^Footnote19, ^Footnote20

Alignment with Priorities and Strategic Outcomes

The objectives of CARA are aligned with federal government priorities relating to air quality, climate change and sustainability as outlined in federal policy and budgetary commitments, as well as commitments made in the FSDS.

• There is alignment of CARA objectives with stated priorities outlined in the 2013 Speech from the Throne, including Government commitments to “take further action to improve air quality nationwide” and to “work with provinces to reduce emissions from the oil and gas sectors while ensuring that Canadian companies remain competitive”.
• Budget 2013 makes reference to CARA, noting that the government of Canada has made available more than $1 billion to support CARA since 2006. The budget also restates Canada’s targets regarding GHGs, including making a commitment to reduce Canada’s total GHG emissions by 17% from 2005 levels by 2020 as set forth in the Copenhagen Accord under the UNFCCC.
• CARA is an important component of the FSDS, which presents the federal government’s environmental priorities and corresponding actions. Now in its second cycle (2013-2016), “Addressing Climate Change and Air Quality” is one of four overarching themes identified in the FSDS. The goals of this theme include references to key deliverables of CARA.

CARA is aligned to the strategic outcomes of EC and HC. Linkages with NRC’s strategic outcomes are appropriate to NRC’s targeted role in CARA.

• The table below summarizes the alignment of CARA activities to the departmental strategic outcomes for each of the participating departments. For EC and HC, CARA activities align with outcomes related to the protection of the environment and human health. The alignment of CARA within NRC reflects its role in supporting Canadian firms who are developing and deploying technological solutions to improve indoor air quality.

Consistent with Federal Roles and Responsibilities

CARA is consistent with federal responsibilities as outlined under CEPA 1999 and there are several Acts that support the roles and mandates of EC and HC in CARA, as well as participation in international agreements.

Legislative Mandate

• Clean air and the regulation of pollution, emissions and radiation is an area of shared P/T and federal government responsibility.^Footnote21
• CARA contributes to fulfilling the federal government’s legislated responsibilities under CEPA 1999, as well as related legislation such as the Department of the Environment Act, in order to prevent and manage the health and environmental risks posed by toxic and other harmful substances.^Footnote22
• CARA is an important component of the FSDS, which presents the federal government’s environmental priorities and corresponding actions. Now in its second cycle (2013-2016), “Addressing Climate Change and Air Quality” is one of four overarching themes identified in the FSDS. The goals of this theme include references to key deliverables of CARA.
• The Minister of the Environment is accountable to Parliament for the administration of all of CEPA 1999. Under CEPA 1999, the Ministers of the Environment and Health jointly administer the task of assessing and managing risks associated with toxic substances, including those arising from climate change and air pollution.
• The Cabinet Directive on Regulatory Management assigns federal departments and agencies responsibility for cooperating with P/T governments in developing and implementing regulations.

International Commitments

• CARA supports the federal government in addressing a number of international commitments and fulfilling reporting obligations (discussed in more detail in Section 4.2.2, outcome ii), including:
  • the Canada-US Air Quality Agreement (AQA);
  • a number of protocols under the UNECE LRTAP Convention;
  • the Minamata Convention on Mercury;
  • the United Nations Framework Convention on Climate Change (UNFCCC);
  • commitments made in the 2009 Copenhagen Accord to achieve a 17% reduction in GHG emissions below 2005 levels by 2020; and
  • plans announced in May 2015 to reduce GHG emissions by 30% below 2005 levels by 2030 and to support progress towards a new global climate change agreement under the UNFCCC.^Footnote23

Given the shared responsibilities between the federal and P/T jurisdictions, mechanisms are in place or planned for multilateral and bilateral collaboration with the provinces/territories, which will help mitigate potential duplication.

• The federal government is engaged in efforts to collaborate with provinces and territories in a number of ways, including:
  • Multilateral discussions through the Canadian Council of Ministers of the Environment (CCME) which supports the development of a nationally consistent approach to air pollution and endorsement of the AQMS;
  • Establishment of the Mobile Sources Working Group (part of the work related to AQMS under the CCME) to address emissions from mobile sources; Partnership with Alberta to deliver the Joint Canada/Alberta Implementation Plan for Oil Sands Monitoring;
  • Establishment of equivalency agreements to reduce duplication between federal and provincial regulations with equivalent environmental outcomes;
  • The AQHI Implementation Committee comprised of health and environment representatives from various levels of government; and
  • Bilateral collaboration on P/T radiation protection programs and multilateral cooperation through mechanisms such as the Federal Provincial Territorial Radiation Protection Committee to support radon outreach, testing and mapping efforts.

4.2 Performance Efficiency and Economy

CARA’s design of gives it the appropriate capacity and flexibility to respond to an evolving policy direction and advance the government’s regulatory agenda. Processes and tools are in place to help determine the most suitable package of measures (regulatory or other non-regulatory instruments) to address air pollution and GHG emissions.

• The majority of internal interviewees noted that the design of CARA consisting of five activity elements was appropriate, commonly describing it as "sensible" and comprising the key building blocks for a flexible, rigorous and risk-based response.
• The EC instrument choice framework helps regulators identify the most appropriate tool for achieving health and environmental benefits based on the intended results, a competitiveness analysis, and a sector/facility/technology profile. Under CARA, we are offered examples of the different approaches and tools (i.e., regulations, standards, codes of conduct, guidelines, pollution prevention plans) being implemented.
• An important consideration in CARA’s design is the need to ensure sufficient flexibility and capacity to respond to evolving government direction. For the most part, the existing design has proved capable of accommodating this requirement. For example, policy analysis has played a key role in ensuring that transportation regulations under CARA are aligned with those in the US. Other regulations have required analyses that take into account Canada's unique emissions profile and the competitive position of Canadian firms. Some key internal interviewees, however, identified the temporary nature of CARA funding as posing challenges with respect to recruiting and retaining qualified policy and scientific specialists and offering the necessary stability to support longer-term planning of science, reporting, policy and regulatory development.

While CARA is a complex, multi-faceted initiative, roles and responsibilities are clear and well-understood. Formal management and governance occurs at various levels, and a number of structures and mechanisms are in place to oversee and coordinate components under CARA. Opportunities for strengthening linkages between regulatory development and reporting activities in order to support harmonization and improve resource planning were nonetheless identified.

• There is no formal CARA-wide governance structure; rather, there are distinct governance bodies for managing initiatives within and/or across each of the five CARA activity elements.
• There are senior-level committees in place to ensure that CARA-related activities are coordinated across government agencies (e.g., Deputy Minister level committee on Climate Change, Energy and Environment for interdepartmental coordination).
• Across government jurisdictions, the Environmental Planning and Protection Committee under the CCME provides for intergovernmental coordination of environmental issues, including air quality, through, for example, the CCME Air Management Committee and Mobile Sources Working Group.
• Program documentation clearly identifies the responsible organization(s) for each of the numerous CARA initiatives within the CARA activity elements and sub-elements. The majority of internal interviewees confirmed that the roles and responsibilities of the various CARA stakeholders are clearly defined and understood, with no significant areas of confusion.
• Linkages among the five CARA elements have been well defined for most inter-related components. Internal interviewees often described the linkages as successful "at the working level" or fostered informally among colleagues with aligned professional interests. Examples of effective linkages include:
  • for science activities, collaboration between EC’s Meteorological Service of Canada and the S&T Branch so as to promote information sharing and identify areas for joint work-planning on air quality, and between EC and HC in order to define health- and environment-based standards for CAAQS, develop BLIERs and implement the AQHI; and
  • for air pollutant regulatory development activities, cooperation between EC and HC science to support and strengthen RIASs.
• Some internal and external interviewees recommended that information sharing and planning between regulatory development activities and reporting be strengthened in order to more fully integrate reporting expertise into the design of reporting requirements for new regulations and ensure that new reporting requirements optimize opportunities for harmonization with existing regulatory reporting and inventories and reduce duplication.
  • Reducing duplication is consistent with the federal directive on streamlining regulation^Footnote24 and also constitutes a priority for regulatees. Addressing duplication was also among the top five suggestions for improvement identified by survey respondents in an open-ended question (identified by 7% of respondents). In addition, both internal and external stakeholders (industry and provinces) pointed to the need for the federal government to clarify its commitment to further expand/adapt the current SW system in order to accommodate new reporting requirements under CARA and other federal and provincial programs.
• A few internal interviewees also identified enhancing communications between policy / regulations and science as a potential area for improvement, noting that this would strengthen the alignment of scientific research with regulatory development activities and ensure that any decisions to regulate would have both a scientific and a policy basis. Initiatives are in place in both EC and HC to strengthen collaboration in this area.

Many aspects of CARA support efficient delivery, including partnering, processes to prioritize the expenditure of funds, and efforts to reduce the cost of consultations through the use of technology.

• Internal interviewees generally feel that work under CARA is being implemented efficiently and collectively identified the following as contributing to efficiency:
  • regulatory alignment with the US;
  • rigorous prioritization of science resources and research funding to address needs; and
  • use of risk management and prioritization to deploy compliance promotion resources for highest impact.
• Another important contribution to efficient delivery concerns the partnering and leveraging of resources, expertise and information with, for example:
  • federal CARA partner departments and branches (e.g., S&T Branch leveraging support from the MSC in support of the Canadian Air and Precipitation Monitoring Network (CAPMoN) delivery);
  • provinces and territories (e.g., shared cost of the single window reporting system, and the use of National Air Pollution Surveillance (NAPS), a long-term ambient air quality monitoring program managed by EC and delivered via a cooperative agreement with the provinces, territories and some municipal governments to support the development of and reporting on CAAQs, the AQHI and the Canada-US AQA);
  • NGO partners (e.g., doing outreach, promoting and endorsing the AQHI and National Radon programs, leveraging their outreach expertise and networks, and lending credibility with key target audiences);
  • Universities, via contribution agreements (e.g., accessing unique scientific expertise and their ability to advance knowledge in support of CARA priorities);
  • industry (e.g., input and testing of single window platform, collaboration with the construction sector to develop and assess new radon measurement and reduction solutions); and
  • the US (e.g., collaboration between EC and the US Environmental Protection Agency under the AQA that leverages scientific research and testing expertise).
• Efforts are also being made to stage consultations in support of the regulatory development process in a cost-effective and efficient manner. Internal interviewees noted that fewer in-person meetings were being scheduled in favour of technology-based or virtual interface meetings. The vast majority of surveyed working group participants (81%) agreed somewhat or strongly that a mix of in-person, teleconference and electronic methods was appropriate in engaging the group. And surveyed working group participants offered positive feedback on the organization of the consultation process, with 77% agreeing somewhat or strongly that the working groups they had participated in were well-managed. These views were also echoed in interviews with external stakeholders, who generally felt that the process was well-supported and efficient.

Performance measurement is occurring, and progress on key aspects of CARA is being reported through a number of public reporting initiatives. Performance measurement plans are also in place at the level of individual regulations.

• Performance reporting for CARA occurs through Departmental Performance Reporting and, more recently, through the FSDS Progress Report. The Progress Report provides high-level information on the progress being made in reducing emissions of air pollutants and GHGs.
• A Performance Measurement Framework that identifies indicators for outputs and outcomes from the CARA logic model was developed in 2011.^Footnote26 Although reporting against these indicators is not systematic, there are numerous other public reporting initiatives associated with CARA that provide extensive performance data on CARA initiatives, including:
  • Annual GHG emissions projections in Canada’s Emissions Trends Report (2011, 2012, 2013, 2014);
  • National Inventory Reports on GHGs and Overview Reports on facility emissions (annual);
  • National Communications (quadrennial) and Biennial Reports under the UNFCCC, including Canada’s Sixth National Communication on Climate Change (2014);
  • National Air Pollutant Emission Inventory (annual);
  • Black Carbon Emission Inventory (new in 2015);
  • Canada-US AQA Progress Reports (biennial-2010, 2012); and
  • Publicly accessible air quality monitoring data via the Open Data Portal of the Canadian Government.^Footnote27
• Reporting also occurs at the level of individual regulations. Various vehicle and engine emissions regulations include end-of-model-year reporting, and a Performance Measurement and Evaluation Plan (PMEP) was developed for the draft MSAPR (published in Canada Gazette Part I in 2014). As the MSAPR regulations come into effect, reporting will focus on the percentage of companies that meet their regulated air pollutant emission reduction requirements. Other regulations with PMEPs include the Heavy-Duty Vehicle and Engine Greenhouse Gas Emission Regulations and the Reduction of Carbon Dioxide Emissions from Coal-fired Generation of Electricity Regulations.
• Some challenges were noted regarding the measurement of longer-term achievements in behavioural change (i.e., for AQHI) and the attribution of decreases in air pollutant and GHG emissions to CARA activities, given the involvement of other jurisdictions and other complicating factors.

4.3 Performance-Effectiveness

As previously described, CARA’s performance was assessed against the eight immediate/intermediate outcomes presented in Annex B of this report. Findings associated with stakeholder collaboration and consultation pertain to outcomes relating to both air pollutant and GHG instrument development (outcomes v and vi) and are presented only once, under outcome vi.

Outcome i: Increased scientific knowledge about indoor and outdoor air quality conditions, greenhouse gases and health and environmental risks that informs policy, regulations/standards and information to Canadians. - Acceptable

An assessment of progress toward CARA’s three final outcomes is also provided. As noted in the limitations section, progress on longer term outcomes cannot be solely attributed to federal efforts under CARA, given the involvement of other jurisdictions.

CARA science has been conducted to address a broad range of air quality and climate change issues and has contributed to a greater knowledge of current and emerging issues both domestically and globally. CARA science has supported policy analysis, regulatory and standards development, and service delivery and has informed federal stakeholders working on transboundary issues and international negotiations. CARA science is conducted in a collaborative manner and is recognized by internal and external stakeholders to be of high quality.

• CARA science, which integrates air quality research, monitoring and modelling, has the objective of quantifying priority air pollutants, identifying trends, predicting air quality both nationally and globally and in the near and distant future, and providing new knowledge and a greater understanding of pollutant exposure on human health and pollutant impacts on the environment. In addition, atmospheric monitoring and modelling of greenhouse gases and aerosols were conducted as part of an integrated climate research program. This program supports Canada’s domestic and international commitments, furnishes information for assessing GHG emissions sources and trends, and supports the development of climate modelling tools. Examples of CARA science contributions include:
  • HC’s Census Cohort Studies have delivered the first Canada-specific analysis of the effects of long-term, low-level exposure of PM_2.5 on the Canadian population.
  • A number of scientific studies were conducted to increase our knowledge of the effects of various air pollutants, including PM, ozone, VOCs and mercury, on air quality, human health and ecosystem health, the results of which were used to improve air quality management.
  • Conducting research to update the AQHI and improve prediction models, for example, through the integration of real-time forest fire emissions, and to develop appropriate health messaging relating to air quality conditions for specific at-risk groups (e.g., asthmatics, persons with cardiovascular disease).
  • Developing the second generation Canadian Earth System Model (CanESM2) which models climate change to make improved projections of the future climate in Canada and globally.
  • Helping to assess the effectiveness of air pollution policies, standards and regulations, including through evaluation of costs and benefits (e.g., vehicle and engine emissions testing and research, health-based intervention studies).
  • Conducting research under the National Radon Program into the risks and prevalence of radon across Canada and the interrelationship between thoron and radon levels; doing residential and field research studies on measurement and mitigation solutions for informing the development of outreach materials, guidance documents, national radon mitigation standards and revisions to the National Building Code of Canada; and working with industry to develop a custom laboratory to evaluate innovative indoor air quality solutions and technologies (The Radon Infiltration Building Envelope Test System).
• CARA science has also helped address emerging and unanticipated issues like the emissions associated with the extraction of shale gas/hydraulic fracturing, short-lived climate pollutants (SLCPs), and a new focus/level of effort on the oil sands region.
• According to documents and interviews, a collaborative approach has been employed within and across federal partner departments whereby the departments meet to discuss research and scientific activities aimed at addressing the needs of CARA.
• Internal and external interviewees expressed favourable opinions on the quality of CARA science. Moreover, an expert panel review conducted in 2012 on the air quality research program carried out by Health Canada’s Healthy Environments and Consumer Safety Branch (HECSB) concluded that HECSB’s work in this area is “efficient, effective, productive and of high quality”.^Footnote28

Outcome ii: Data collection and reporting of GHG emissions and air pollutants that meets and/or exceeds domestic and international reporting requirements, while streamlining burden for sector organizations/industry. - Acceptable

CARA reporting activities allow EC to meet legally binding domestic and international obligations to report on GHGs and air pollutants, support the development of policy and regulations, and inform Canadians.

• Internal interviewee feedback and documentary evidence show that national inventories^Footnote29allow Canada to meet its obligation to: submit the annual GHG Inventory covering anthropogenic emissions by source and removals by sinks in accordance with the UNFCCC; report annually to the UNECE on air pollutants under the LRTAP Convention; and report biennially on progress in meeting commitments under the Canada-US AQA.
• In addition, these inventories support the development and effective implementation of clean air regulations and the implementation of the AQMS by federal, provincial and territorial governments and facilitate monitoring of progress on Canada’s commitment under the Copenhagen Accord.
• All facility and inventory data are made available to the public on EC’s web site. National Pollutant Release Inventory (NPRI) data are made available to the public through an online query site, annual summary reports, emission density maps for key air pollutants and a map layer that allows Google Earth™ users to view the location of facilities that report to the NPRI, along with their pollutant release, disposal and recycling information for the latest reporting year.

The single window (SW) system has been expanded and improved, resulting in a reduced reporting burden on users.

• During the period under study, the SW system expanded from collecting data for two programs to addressing data collection on emissions and other toxins for eleven provincial and federal programs^Footnote30. Agreements have recently been renewed for five years with British Columbia, Alberta and Ontario, and a new agreement has been signed with New Brunswick. System users have also increased from 900 users in 2010 to 20,000 users in 2013. In 2012, 66% of facilities that reported GHG emissions federally also reported emissions under a provincial regulation based on the SW system.
• EC is currently in discussions with Nova Scotia, Newfoundland, Saskatchewan and Manitoba regarding use of the system, and the SW or an alternative electronic reporting system is being considered for use by other CARA and non-CARA EC initiatives (AQMS/BLIERs, coal-fired electricity regulations, renewable fuels regulations, off-road diesel engine regulations, amended environmental emergencies regulations, PCB on-line reporting, pollution prevention plans, and the Canadian Notice and Manifest Tracking System).
• According to internal and external interviewees and case study evidence, the SW system is helping reduce the reporting burden, since tombstone data is only entered once, data is transferred among programs supported by the system, and there has been some harmonization of reporting requirements have been harmonized to some extent. The system improvement cycle, whereby stakeholders submit suggestions and participate in beta testing, has led to decreases in the number of fields that need to be completed. One industry representative estimated that data entry time had been reduced by 50% as a result of streamlined reporting under the system.
• As noted previously, despite the considerable progress made in this area, both internal and external stakeholders (industry and provinces) pointed to the need to continue expanding/adapting the system in order to accommodate new reporting requirements under CARA and other federal and provincial programs.

Outcome iii: Increased access and awareness among Canadians of the Air Quality Health Index (AQHI) and delivery of information on the environmental and health impacts and mitigation strategies related to outdoor air quality. - Acceptable

The number of communities where AQHI is delivered has risen over the evaluation period and is continuing to increase.

• In 2010-11, the AQHI was available in 27 communities that represented approximately 38% of the Canadian population.^Footnote31 As of September 2014, the AQHI was available to 69% of the Canadian population^Footnote32 in 84 communities in 10 provinces (with pilot projects in Ontario and Quebec) and one territory.
• Availability will continue to increase, as plans are in place to implement the AQHI in Ontario, Nunavut and the Yukon in 2015-16 and discussions with Quebec regarding a complementary air quality forecasting service are still ongoing.
• Program staff indicated, however, that the target of reaching at least 90% of Canadians by March 31, 2016, may be overly ambitious, given the work that still needs to be done to reach remote and northern areas. Simulated science-based values from air quality modelling (and possible contributions from satellite-derived air quality concentrations) are expected to support the implementation of AQHI where air quality monitoring sites cannot be deployed in a cost-effective way. The methodology is expected to be in place for 2018.

P/Ts, NGOs and other partners are conducting targeted communication campaigns and innovative marketing strategies, which are expected to increase awareness of the AQHI.

• A number of outreach initiatives have been undertaken with a view to boosting awareness of AQHI. These include:
  • various HC AQHI Program resources (e.g., social media, posters, brochures);
  • engagement activities by Regional Air Specialists (e.g., events championing the AQHI involving athletes and other outdoor enthusiasts); and
  • integration into partner programs and information campaigns (e.g., Toronto Public Health, Asthma Society of Canada, York Region Public Health).
• The AQHI program has made strategic use of its limited outreach funding by developing long term relationships with partner organizations (NGOs and community groups, provincial environment and health ministries, and municipal public health authorities) in order to develop and conduct communication campaigns across the country that target, among others, sensitive and at-risk populations (eg, children, seniors, and people with heart or respiratory conditions). Examples of recent innovations in outreach include:
  • use of geographic information systems to identify target populations and help develop tailored messaging;
  • use of social media and e-applications to develop customized and regularly updated messaging;
  • an on-line learning course for health professionals in partnership with the University of British Columbia; and
  • a joint initiative with the College of Family Physicians of Canada aimed at recruiting physician AQHI ‘champions’ from across the country to raise awareness among health professionals of the link between health and the environment, in particular outdoor air quality. The campaign reached almost 6,000 health professionals.^Footnote33

Outcome iv: Ongoing collaboration with the US on reduction of transboundary air pollution, including the consideration of a particulate matter annex to the Air Quality Agreement. - Acceptable

Since 1990, through cooperation with the US, there has been a reduction in the transboundary movement of air pollutants that cause acid rain and smog. On the issue of PM, ambient concentrations of PM_2.5 have diminished significantly along the border. Recent analyses have concluded that the addition of a PM annex to the AQA is not currently required, although discussions are continuing between the countries on how best to address PM within the agreement.

• The transboundary movement of air pollutants is reduced and reported on through cooperation with the US under the AQA. The AQA is a 24 year old agreement that originally committed both countries to reducing acid rain-causing emissions of SO₂ and NOx. It was amended in 2000 to further address emissions of NOx and VOCs, which are the precursors to ground-level ozone, a key component of smog.
• There have been significant decreases in the transboundary movement of air pollutants responsible for acid rain and smog. The total SO₂ emissions of Canada and the US have decreased 60% and 78%, respectively, from 1990 levels, and their emissions of NO₂ have fallen by 28% and 48%, respectively, over the same period.^Footnote34
• In 2007, Canada and the US jointly announced their intention to seek the addition of an Annex to the AQA to address transboundary PM. Scientific analyses carried out in 2013 to inform a decision on the need for an Annex^Footnote35concluded that ambient concentrations of PM_2.5 had diminished significantly in both countries in regions near the border, and modelling of future air quality conditions suggested that ambient air quality standards for PM_2.5 in both countries would not be exceeded (except in a few locations in southern Ontario and Quebec owing to local emissions sources). As a result, the findings of the analyses did not support the addition of a PM Annex to the AQA at the time.
• Both Canada and the US have agreed to consider the possibility of incorporating commitments to further reduce PM emissions and/or its precursor emissions in future iterations of the AQA, as exposure to PM_2.5 can have negative public health effects even at or below current standards. Interviewees expect that future AQA discussions will also consider process (reporting) improvements, enhanced science / technology exchanges, and the assessment of transboundary PM movements on a more regional scale. This may include such considerations as the impact of oil sands development and other non-conventional gas and oil extraction/production on PM in the central and western parts of the two countries, as well as the effect on PM and black carbon of development in the Arctic.

Outcome v: Health-and environment-based ambient air quality standards and BLIERs (industrial emission standards) are developed collaboratively and endorsed by all jurisdictions. - Acceptable

There has been considerable success in securing broad endorsement for the development of ambient air quality standards and BLIERs through the establishment of the AQMS. These in turn are projected to lead to reductions in air pollutants from regulated sectors and to provide high net benefits from positive health and environmental impacts.

• In 2012, the CCME agreed, with the exception of Quebec,^Footnote36 to implement the AQMS. This comprehensive national approach to improving air quality in Canada --developed through a multi-stakeholder process that included federal, provincial and territorial governments, industry, and non-governmental and aboriginal organizations--includes BLIERs for key industrial sources and new health- and environment-based CAAQS.
• The work undertaken as part of the AQMS includes:
  • New, more stringent CAAQS for PM_2.5 and ground level ozone that focus on continuous improvement were endorsed by Environment Ministers under the AQMS in 2012^Footnote37 and published in Canada Gazette Part I in 2013;
  • The development of CAAQS for other pollutants of concern, such as SO₂ and NO₂, which is currently underway; and
  • Analyses for the establishment of BLIERs covering 13 industrial sectors, 3 equipment groups and 1 cross-sectoral pollutant source.^Footnote38
• BLIERs are being implemented using a mix of regulatory and non-regulatory instruments. Regulated BLIERs will be incorporated into one regulation, namely MSAPR. The MSAPR was published in Canada Gazette Part I in June 2014 and covered 3 BLIERs: cement production, non-utility industrial boilers and heaters, and natural gas-fired reciprocating (stationary) engines. The expected air pollutant reductions stemming from this regulation are 2065 kilotonnes (kt) of NOx and 96 kt of SO₂ between 2013 and 2035.^Footnote39Regulations for the remaining BLIERs are expected to be developed in phases and implemented through amendments to MSAPR.
• To assess the potential impact of a regulatory initiative, EC economists and scientists work with partners at HC to prepare a RIAS. The estimated costs and benefits of the regulation are articulated from a societal perspective and from the point of view of regulatees. Regulations under CARA envisage a high benefit-to-cost ratio. For the draft MSAPR, for example, the net benefits accruing to Canadians from the regulation of emissions for the period 2013-2035 in terms of health and well-being are estimated to be $1.4 billion from the cement manufacturing sector, $1.1 billion from boilers and heaters and $6.5 billion from gas-fired engines. These numbers are based on the present values associated with avoided premature deaths, hospital visits, asthma episodes, sick days and other illnesses, and other environment-related issues.
• External stakeholders view the establishment of the AQMS as a success, noting that the CCME created a forum for strong federal, provincial and territorial cooperation that facilitated the implementation and refinement of the System, with broader input via the Stakeholder Advisory Group. Some stakeholders, however, feel that owing to this collaborative forum the BLIERs represent a less ambitious consensus on industrial emissions than might have been reached if these standards had been developed through a less collaborative process.

Independent of the AQMS and its associated measures, transportation sector regulations have also been developed under CARA that align with US and non-regulatory instruments to help reduce air pollution emissions.

• Transportation sector regulations are generally considered by stakeholders to be a ‘success story’ in fulfilling the commitments made under CARA and in harmonizing this sector’s regulations with those in the US. Transportation regulations that address air pollution include:
  • Marine Spark-Ignition Engine, Vessel and Off-Road Recreational Vehicle Emission Regulations - published in Canada Gazette Part II in February 2011, and applicable as of the 2012 model year.
  • Regulations Amending the Off-Road Compression-Ignition Engine Emission Regulations - published in Canada Gazette Part II in December 2011.
  • Regulations Amending the On - Road Vehicle and Engine Emission Regulations (On-Board Diagnostic Systems for Heavy-Duty Engines and Other Amendments) - published in Canada Gazette Part II in February 2013.
  • Proposed Regulations Amending the On-Road Vehicle and Engine Emission Regulations (Tier 3 vehicle standards) - published in Canada Gazette Part I in September 2014 and subsequently finalized and published in Canada Gazette Part II on July 29, 2015; and
  • Proposed Regulations Amending the Sulphur in Gasoline Regulations (Tier 3 fuel standards) - published in Canada Gazette Part I in September 2014 and subsequently finalized and published in Canada Gazette Part II on July 29, 2015.
• The non-regulatory instruments developed to date notably include the Codes of Practice. These have already been published for comment for the aluminum/alumina and iron/steel sectors (June 2014), and a Code of Practice to reduce emissions of VOCs from Cutback Asphalt is planned for 2015. Proposed VOC Concentration Limits have also been developed for several types of products (architectural coatings, automotive refinishing products, as well as other miscellaneous products including personal care, automotive and household maintenance products).

Outcome vi: GHG regulations are developed and implemented in collaboration with stakeholders and supported by policy analysis and advice. - Opportunity for Improvement

N.B. The following discussion on stakeholder collaboration/consultation also pertains to outcome v on the development of air quality standards and BLIERS for air pollution.

Consultative mechanisms have been used under CARA to develop regulations, standards and non-regulatory instruments governing both air pollutants and GHGs. Stakeholders generally view the federal approach to consultations as a strength, though they did identify opportunities for improvement.

• In line with the Cabinet Directive on Regulatory Management calling on federal regulators to consult with stakeholders, working groups consisting of representatives of P/T governments, industry, NGOs and other groups were established under CARA and tasked with contributing to the regulatory development process. During the period under study, over 1,100 individuals participated in 57 working groups focusing on air pollutants and GHGs (almost 30% of these individuals participated in more than one working group during the period under study).
• The composition and methodology of these groups varied according to the nature of the sector, the purpose of the group and the stage of regulatory development. While these consultations were seen to consume significant amounts of time and resources, internal interviewees saw the investment as worthwhile since they believed that this contributed to a higher quality instrument and greater compliance.
• In response to an open-ended question in the survey of working group members, the respondents most commonly identified the federal government’s consultative approach as the key element in the federal efforts to address air pollutant and GHG emissions (mentioned by 49%). Moreover, 76% of those surveyed agreed somewhat or strongly that the working group was an effective way to obtain input from stakeholders. (Higher levels of agreement (88%) were reported by those involved in the CAAQS working groups).
• A majority of surveyed external stakeholders also agreed that the group was effective in accomplishing objectives to a large or great extent (67%) (though the agreement among those involved in oil and gas working groups was 48%). Roughly half of the members of groups that had completed the development of a regulatory or alternative instrument believed to a large or great extent that the working group’s efforts had produced an instrument that was timely (51%) and practical (47%).^Footnote40
• Areas of the consultation process that required improvement according to the survey or external interviewees included:
  • a desire for more frequent or earlier consultation with stakeholders;
  • a stronger focus on delivering on objectives within clear timelines;
  • limited inclusion/inability of NGOs to participate in consultations (based on the lists of working group members drawn up for the survey, just under 5% of working group members were from NGOs);
  • a desire to ensure participation by the full spectrum of industry stakeholders;
  • tight timelines for responding to analyses/information given the group; and
  • a lack of consistency in processes across working groups (e.g., protocols on documentation sharing, group management);
• In addition, relatively few members held the view that the instrument developed was evidence-based (informed by science, socioeconomic analysis) (34% of survey respondents). This issue was also raised by a small number of external stakeholders during interviews.

GHG regulatory development has been supported by policy analysis. GHG regulations have come into effect governing the transportation and coal-fired electricity generation sectors. While regulatory development work has occurred for oil and gas, Emissions Intensive Trade Exposed (EITE) and natural gas-fired electricity sectors, these regulations have yet to reach the publication stage.

During the time period under study, policy work in support of GHG regulatory development included:

• a framework to help assess the potential impacts of GHG regulations on economic competitiveness, which underpinned all the competitiveness analysis conducted between 2011 and 2015;
• an Offset System for GHGs which was designed as a flexible compliance instrument for Canada’s sector-by-sector regulatory approach to reducing GHG emissions from industrial sources; and
• research and analysis on the “social cost of carbon,” which is a key element in cost-benefit analysis.

To date, CARA regulations have been put in place to reduce GHG emissions from two of the largest emitting sectors: the transportation sector (23% of emissions) and the electricity sector (12% of emissions).^Footnote41

Transportation Sector

• Passenger Automobile and Light Truck Greenhouse Gas Emission Regulations came into effect in September 2010. Over the operational lifetime of all 2011 to 2016 model year vehicles, it was estimated that the regulations will result in a cumulative reduction of 92 megatonnes (Mt) of carbon dioxide equivalent (CO2e) in GHG emissions.^Footnote42
• Heavy-Duty Vehicle and Engine Greenhouse Gas Emission Regulations were published in Canada GazettePart II in March 2013. It was estimated that the regulations will result in a reduction of approximately 19.1 Mt of CO2e in GHG emissions over the lifetime of vehicles produced during the 2014-2018 model year.
• Regulations Amending the Passenger Automobile and Light Truck Greenhouse Gas Emission Regulations were published in Canada Gazette Part II in October 2014. It is projected that the regulations will deliver total GHG reductions of 174 Mt over the operational lifetime of all 2017 to 2025 model year vehicles sold in Canada.
• Also in October 2014, the federal government published a Notice of Intent to develop regulations to further reduce GHG emissions from on-road heavy-duty post-2018 model year vehicles and engines, in alignment with US standards.

Electricity Sector

• Reduction of Carbon Dioxide Emissions from Coal-Fired Electricity Generation Regulations were published in September 2012, setting a performance standard for new units and units reaching the end of their useful lives. The performance standard in these regulations came into force on July 1, 2015, and is expected to result in a net cumulative reduction in GHG emissions of roughly 214 Mt over the period 2015 to 2035. Plans to build on this standard with regulations for natural gas-fired electricity were announced in May 2015.^Footnote43

At the present time, there are no regulations for the oil and gas and EITE sectors, two other large emitting sectors (25%, 11% of emissions respectively).^Footnote44

• The oil and gas sector working groups are currently on hold, although internal analysis is ongoing and a draft regulatory package has been developed. A plan to introduce regulations to reduce methane emissions from the oil and gas sector in alignment with the US was announced in May 2015.^Footnote45
• As for the EITE sectors, working groups for various sectors have been established, and consultations and regulatory development are ongoing. Regulatory development has seen the most progress in the chemicals, nitrogen-fertilizers (covering ammonia and nitric acid) and aluminum EITE sectors. In May 2015, the Government announced its intent to develop regulations for chemicals and fertilizer production.

The lack of progress on regulations to reduce GHG emissions is a cause for concern among many external interviewees and was also highlighted by the CESD.^Footnote46 A great deal of work has been done to advance regulatory development, including regulatory drafting. In some cases, however, there have been delays in publishing regulations, owing to an evolving policy environment. According to documents and internal and external interviewees, one key concern, which relates in particular to GHG emissions regulations for the oil and gas sector, is whether regulations would have a negative impact on the sector’s competitiveness. Some of these interviewees have pointed out that at present there is limited scope for US alignment when it comes to GHG regulations governing the oil and gas sector given that the US has only recently moved to directly regulate GHGs, specifically methane, within this sector (although regulations on VOC emissions have been implemented, which has indirectly resulted in considerable methane co-benefits in the oil and gas sector). This divergence in regulatory prioritization may be due to the different emissions profile of the US, where power generation is deemed a prime area of opportunity for emission reductions given the high concentration of coal-fired generation and, as a result, has received more regulatory attention thus far than the oil and gas sector. In response to a recommendation from the CESD, EC has committed to strengthening its planning process in support of the government’s climate change agenda.^Footnote47

Given that clean air and the regulation of GHG and air pollutant emissions is an area of shared jurisdiction, there is significant provincial interest in equivalency agreements for regulations under CEPA 1999 that would mitigate duplication, though there is limited experience with these complex agreements to date.

• Evidence from documents, internal and external interviewees and a case study indicate that there is considerable provincial interest in using equivalency agreements under CEPA 1999 to avoid potential regulatory duplication. CEPA 1999 permits a federal regulation to stand down if the province or territory has an enforceable regulation with an equivalent or better environmental outcome.
• One equivalency agreement has been finalized to date for the federal coal-fired electricity regulations, ie, with Nova Scotia in 2014.
• Over the past five years, bilateral discussions have taken place with other provinces on the possibility of signing equivalency agreements.
• The communication of clear processes and guidelines will be important as work continues in this area. Some external stakeholders pointed to a lack of clarity regarding the processes and circumstances under which equivalency agreements will be established.
• Internal interviewees anticipate increased interest in these agreements. Depending on how things work out in this area, the demand for resources to support the development of equivalency agreements may increase (e.g., policy analysis and scientific modeling), particularly when the range of possible combinations of provinces and regulations is considered.

Outcome vii: Regulatees understand and comply with their obligations related to reporting requirements and GHG and air pollutant emissions. - Acceptable

Participation in compliance promotion/information sessions and regulatory development consultations, plus clearly defined technical requirements, have helped ensure that, overall, regulatees clearly understand their emissions and reporting requirements.

• Compliance promotion must appeal to a variety of methods for ensuring awareness and understanding and responding to inquiries (e.g., information sessions/meetings, social media, emails and mail outs, technical guidance documents). According to the survey results, approximately two-thirds (64%) of industry working group participants impacted by an existing CARA regulation reported that their organization had attended a compliance promotion or information session on regulations or non-regulatory instruments for their sector. A large majority of those who attended agreed somewhat or strongly that the sessions had been convenient (77%), informative and useful (74%), and that they had provided clear, comprehensive and useful information (72%); and had been staged in a timely manner to allow organizations to take the necessary action (66%).
• Internal and external interviewees indicated that they felt that regulatees understood their obligations, owing in part to the early involvement of stakeholders in regulatory development consultations.
• Of the surveyed working group members who are regulatees:^Footnote48
  • Over two-thirds agreed somewhat or strongly that the regulations impacting their organization had clearly described technical requirements (definitions, prescribed limits, determination of actual emissions and sampling requirements) and that monitoring and reporting requirements were clear (68% and 67% respectively).
  • While the majority offered positive ratings, a smaller number of regulatees agreed that sufficient time had been provided to prepare for compliance (58%), that the requirements had been clearly communicated to them (55%), and that the reporting requirements were manageable (51%).
• Compliance with transportation sector and consumer products regulations is assessed by means of sample testing and regulatee self-reporting. CARA funding has been used to develop EC scientific expertise and infrastructure for the purpose of, among other things, enhancing existing vehicle and engine testing capacity in support of administration and compliance testing for new transportation sector regulations and VOCs regulations for certain products. EC collaborates with the US Environmental Protection Agency through joint work plans on vehicle and engine emissions testing and verification, although a few internal interviewees felt that additional resources were required to support an adequate sampling of vehicles/engines for compliance testing.
• The 2012 EC Evaluation of the Transportation Sector Emissions Program concluded that regulatees generally understand their obligations under the regulations and adhere to those obligations. The assessment of compliance with vehicle and engine emission limits is a relatively complex exercise given the combination of fleet average and individual vehicle air pollutant standards and the size and high turn-around rate of the regulatee community, not to mention increases in the number of new small importers. Nevertheless, the available information shows general compliance with the fleet average standard for NO_x emissions under the On-Road Vehicle and Engine Emission Regulation. Non-compliance with the regulations is often found among new importers of products from emerging markets; however, these represent a comparatively small percentage of product imports.
• An electronic reporting system is in place for the Passenger Automobile and Light-Truck Greenhouse Gas Emission Regulations. Although it is still too early to fully assess this regulation, initial information indicates that, overall, companies generated emissions credits, indicating that the average GHG emissions performance for the fleet was below the regulated standard.^Footnote49
• CEPA and the transportation sector regulations include administrative provisions requiring companies to report emissions-related defects along with their proposed corrective measures. Over the five-year period 2010-2014, the Department received reports on nearly one million vehicles and engines.

Outcome viii: Increased access to information and awareness among Canadians of the environmental and health impacts and mitigation strategies related to indoor air pollution. - Acceptable

Access to information and levels of awareness on indoor air quality -- in particular the risks associated with radon exposure -- have increased.

• The National Radon Program is a key component of the IAQ activity element.^Footnote50 The program includes a scientific research and testing component and provides for the development and distribution of products and outreach programs aimed at raising the awareness among the public and key stakeholder groups of the health risk from indoor radon exposure. HC and NRC scientific collaboration has played an effective role in informing federal policy. For example, NRC and HC research into radon measurement has informed the development of federal, provincial and territorial standards and guidelines on radon.
• The National Radon Program has developed and distributed a variety of outreach products and programs designed to increase awareness and promote action to reduce radon exposure. Examples include:
  • The HC Radiation Protection Bureau, working with Regional Radiation Specialists, leads a variety of outreach and engagement activities, including media interviews, liaisons with stakeholders, contracts with NGOs for collaborative outreach activities, and the staging of radon workshops (including live webinars) and presentations at conferences and tradeshows.
  • In 2013-14, HC attended or organized 113 outreach events, distributed 1,038,000 brochures, responded to 790 public inquiries and had 122,000 web page views, which continued a trend in year-over-year increases observed since 2008. To broaden the reach and impact of its outreach activities, Health Canada employs social media, engages in joint efforts with partners, and supports both a national campaign and an outreach program that targets radon-prone locations and at-risk groups.
• As of 2014, more than 200 individuals across Canada have been certified through the Canadian National Radon Proficiency Program (C-NRPP). C-NRPP certified professionals expand radon awareness and action by providing radon measurement and mitigation services in compliance with federal guidelines.
• The Canadian Household and Environment Survey,^Footnote51 conducted every two years by Statistics Canada, showed an increase in general public awareness of radon between 2011 and 2013 (from 40% to 45%) and further found that the proportion of respondents able to provide an accurate description of radon had increased significantly (from 37% to 53%). HC public opinion research involving targeted households (decision-makers and owners of single family dwellings with a ground floor) also indicated a significant increase in public awareness of radon since 2007 (from 52% of respondents aware of radon in 2007 to 65% in 2013).^Footnote52
• In addition to the National Radon Program, the IAQ element also includes HC work on biological and chemical contaminants aimed at:
  • assessing the risk of indoor air contaminants, including the development of Residential IAQ Guidelines for four priority VOCs;
  • forwarding information to public health officers and environmental health inspectors on health effects and recommended measures for reducing exposure to contaminants in Canadian homes.
  • Health-focused assessments also informed a Memorandum of Agreement with the Standards Council of Canada which established a process for developing health-based voluntary product emission standards, beginning with VOCs in composite wood products used in homes;
  • evaluating the effectiveness of intervention methods for reducing the impact of air pollutants in homes, schools and long-term care facilities (from products/activities associated with attached garages, traffic sources and carbon monoxide); and
  • executing an integrated communications approach to the promotion of key indoor air quality issues via social media, partnerships with retailers and the provision of support to HC regions to engage a broad audience of professionals and vulnerable populations. Between 400 and 500 public inquiry calls or emails are received annually by HC about indoor air quality.

Final Outcome i: Reduced emissions of GHGs and air pollutants from regulated sectors while maintaining competiveness in these sectors - Attention Required (emissions of GHGs) / Acceptable (emissions of air pollutants)

During the time period under study, Canada’s progress on reducing GHG emissions has been limited.

• While national GHG emissions trends reflect Canada’s efforts as a whole and are not solely attributable to the federal government’s efforts under CARA, data over a 15-year period indicate that GHG emissions were higher in 2013 (most recent data available) than in 1990.^Footnote53 GHG emissions decreased by 23 Mt (3%) between 2005 and 2013; however, this decrease was largely due to decreases in 2008 and 2009 that coincided with the global financial crisis. GHG emissions have increased slightly each year from 2009 to 2013,^Footnote54 although current trends may still reflect post-2008 economic adjustments. Overall emissions decreases are largely attributable to the electricity sector (36 Mt or 30% decrease) and EITE industries (13 Mt or 15% decrease), while increases in emissions were recorded for the oil and gas sector (22 Mt or 14% increase).^Footnote55
• The intensity of Canada’s emissions (GHG emissions relative to Gross Domestic Product) has improved by 31% since 1990, however. The more recent 2005 to 2013 time period also reflects this trend, although data in 2013 suggest that emissions intensity has now stabilized.^Footnote56Likewise, per capita emissions show similar trends. As one of the highest per capita emitters globally, Canadians released 22.1 tonnes of GHGs per capita in 1990. In 2005 this indicator had risen to 23.2 tonnes; by 2009, however, it had dropped to 20.8 tonnes and has remained at historic lows ever since.^Footnote57
• Canada’s Emissions Trends report for 2014 forecasts that Canada’s GHG emissions will be 727 Mt in 2020. While this represents a reduction in emissions of 130 Mt by 2020 as compared to a “without measures” or “business as usual” scenario, further reductions of 116 Mt will be required to meet Canada’s Copenhagen commitment.^Footnote58 Emissions from the oil and gas and buildings sectors are projected to increase (by 45Mt and 14Mt respectively) between 2005 and 2020, while emissions in the electricity sector are projected to decrease over the same period (by 50Mt). Emissions in other sectors - transportation, EITE, agriculture, waste and other sectors - are projected to remain stable compared to the 2005 baseline.
• As noted in the discussion of outcome vi, Canada’s lack of progress in reducing GHG emissions was highlighted by the CESD; in response to a CESD recommendation, EC has committed to making additional efforts in this area.^Footnote59

Industrial emissions of smog forming air pollutants have declined steadily since 1990. Particulate matter emissions have for the most part remained stable since 2000, although decreases in southern Ontario and the US have resulted in significant reductions in regional concentrations.

• Air pollutant emissions reductions are the result of multiple actions, including P/T regulations and programs, and municipal and federal actions.
• National emissions steadily declined between 1990 and 2013 for most air pollutants that contribute to smog formation (by 28% to 63% across various types of pollutants).^Footnote60 As noted in the discussion under outcome iv, there have been significant decreases in both Canada and the US in the total emissions of and NO₂since 1990.^Footnote61
• National ammonia emissions, however, increased by 22% between 1990 to 2002, but have since remained stable.
• National fine particulate matter emissions (PM_2.5) increased by 4% betSO₂ween 1990 and 2000, but have remained relatively stable since 2000.

Final Outcome ii: Reduced ambient concentrations of PM_2.5 and ground level ozone. - Acceptable

Ambient concentrations of PM_2.5 have decreased significantly along the Canada-US border. Declines in industrial emissions of smog forming air pollutants contributed to a 15% decrease in national peak ozone concentrations between 1998 and 2012.

• National average and peak outdoor concentrations of fine particulate matter (PM_2.5) have fluctuated over the years with no discernible trends. At the regional level, however, outdoor concentrations of PM_2.5 have decreased in some parts of the country. For example, from 2000 to 2012, annual peak and annual average concentrations of PM_2.5 have decreased by 46% and 36% respectively in southern Ontario owing to decreases in emissions from local sources in southern Ontario and transboundary pollution from the US.^Footnote62
• Scientific analyses completed in 2013^Footnote63 confirmed that ambient concentrations of PM_2.5 have decreased significantly in both countries in regions near the border; modelling of future air quality suggested that ambient air quality standards for PM_2.5 in both countries will not be exceeded (except in a few locations in southern Ontario and Quebec owing to local emissions sources).
• National peak outdoor concentrations of (ground-level) ozone decreased by 15% between 1998 and 2012, owing to reductions of ozone-forming pollutants emissions in Canada and the US.^Footnote64
• While peak outdoor ozone concentrations have decreased, average ozone concentrations have fluctuated from year to year with no discernible upward or downward trend.^Footnote65

Final Outcome iii: Canadians change behaviour to reduce exposure to air pollutants to improve their health. - Unable to Assess (behaviour change as a result of the AQHI) / Acceptable (behaviour change related to radon)

Although some studies have been conducted, there is insufficient information to assess the degree to which availability of the AQHI has led to behaviour changes.

• In an effort to measure awareness and behaviour change, the AQHI program commissioned a series of post-event factual surveys (one in Winnipeg, one in Yellowknife and two in Toronto in 2014). While post event surveys cannot replace a national survey designed to measure AQHI awareness and behaviour change across Canada, they do provide some valuable location-specific insight into the public response to air quality warnings. The response, as measured by the surveys, was found to be highly variable, reflecting the variability of the events themselves and the context in which they take place.^Footnote66
• Data on AQHI awareness and behaviours are also being collected by some partner organizations on a smaller scale, and e-applications are offering promising opportunities to collect outcome data for AQHI. For example, an Ontario Lung Association-sponsored asthma application (Breathe) has collected data showing that AQHI has helped over 50% of its users enjoy a better quality of life. And, though not based on a random sampling of Canadians, data collected from over 600 adult members of the public who completed surveys at AQHI promotional booths at outdoor special events in the City of Hamilton in 2012 found that 67% stated that they were aware of the AQHI, 30% checked the AQHI numbers, and 22% followed AQHI health messages.^Footnote67

There is some evidence that public awareness and knowledge of radon is translating into increases in testing. Construction codes and guidelines are changing to address radon in new buildings.

• According to Statistics Canada data, there was an increase in the percentage of households that had tested for radon between 2009 (3%) and 2013 (5%) as evidenced by an increase in ‘do-it-yourself’ radon testing.^Footnote68 Furthermore, a large US-based radon laboratory shared data with HC showing that the number of radon test kits it had received for analysis from clients in Canada had increased year-over-year by approximately 100% from 2011 to 2012, 60% from 2012 to 2013 and 20% from 2013 to 2014).^Footnote69
• The National Radon Program has put in place mechanisms to influence the behaviour of construction sector stakeholders. New radon protection measures were incorporated into the 2010 edition of the National Building Code that require building designers to take radon protection into account in new construction and to offer flexibility regarding the use of alternative methods of radon protection in new construction. Organizations such as the Canadian Home Builders Association and the Association of Municipalities of Ontario have published information on radon mitigation for their members. HC Regions are working with provinces to incorporate amendments that incorporate radon protection into provincial building codes. And HC is working with the Canadian General Standards Board on the development of two national standards governing radon mitigation in existing and new home construction.

Unintended Outcomes: - Not applicable

No significant unintended outcomes of CARA were identified, either positive or negative.

• A detailed assessment of the World Class Regulator (WCR) criteria was beyond the scope of this evaluation, but evidence suggests that CARA regulations embody the WCR criteria. The evaluation examined best practices in the context of EC’s World Class Regulator (WCR) initiative, namely examining the extent to which CARA regulatory programs incorporate the WCR criteria of being efficient, adaptable, transparent, evidence-based and effective.^Footnote70
• Several of the WCR criteria have been examined in other sections, and the evidence suggests that CARA regulations reflect the WCR criteria, although some opportunities for improvement were identified. In particular:
  • Evidence of “efficiency” was demonstrated through the evidence provided by various mechanisms associated with efficient operations, as well as by the regulations’ high benefit-to-cost ratio.
  • “Adaptability” was demonstrated through the use of various regulatory and alternative instruments and the ability to respond to evolving government policy.
  • With respect to “transparency”, the development of the AQMS and air pollution and GHG regulations arose out of an extensive consultative process involving a significant number of stakeholders convened through various working groups (although some opportunities for improvement were identified).
  • Regarding “evidence-based”, the evaluation found that scientific and socio-economic analysis was conducted in support of regulatory development, although there is room for improvement when it comes to sharing this evidence with stakeholders during the consultation process.
  • Finally, regarding the final criteria of “effectiveness”, the regulations have not been in force for a sufficient period of time for a full assessment.

5.0 Conclusions

Relevance

CARA is highly relevant and addresses a clear and demonstrated need for a continued effort to address the environmental and health effects of air pollution, climate change and poor indoor air quality.

CARA contributes to federal priorities aimed at minimizing environmental threats from air pollution and climate change, managing health risks and ecosystem impacts associated with air pollution, climate change and poor indoor air quality, and contributing to sustainable development.

CARA is aligned with the roles and responsibilities of its partner departments and with the legislative authority of EC and HC under CEPA 1999. CARA also supports the federal responsibility for transboundary pollution and the commitments related to emission reductions and reporting requirements in international agreements.

Performance - Efficiency and Economy

CARA, in its current form, continues to serve as an appropriate and flexible model for providing the capacity and expertise to carry out the federal agenda. Some challenges have been identified, however, with respect to recruiting and retaining qualified policy and scientific specialists and providing the necessary stability to support longer term planning, given the temporary nature of CARA funding.

The current governance mechanism is functioning effectively, and the roles and responsibilities of internal stakeholders are clear and well understood. However, an opportunity was identified to strengthen linkages between the regulatory and reporting elements of CARA and thereby ensure that reporting requirements are considered earlier in the regulatory development cycle. This will provide for better harmonization with existing regulatory reporting processes and improve resource planning for future enhancements.

Several aspects of CARA contribute to efficient delivery, including the use of partnering, processes for prioritizing the expenditure of funds, and efforts to reduce the cost of consultations through the use of technology.

Information on air pollution and climate change is available and publicly reported, although attributing changes in national emissions levels to CARA activities is a challenge. While performance measurement and evaluation plans are in place for some regulations, only limited information is available on their impact, as many regulations have yet to come into effect.

Performance - Achievement of Outcomes

Progress has been made towards achieving many CARA outcomes, including those related to expanding scientific knowledge, increasing ongoing collaboration with the US on the reduction of transboundary air pollution and ensuring that regulatees understand and comply with their obligations.

There is greater access to information and increased awareness of Canadians of indoor air quality issues. On the issue of radon, there is evidence of more frequent home testing, and the practices of the construction sector are evolving through changes to codes and guidelines governing radon.

Coverage of the AQHI has expanded over the study period, and numerous outreach efforts have been undertaken to boost awareness of the AQHI, although only limited data is available to measure behaviour change in response to these efforts.

Data collection and reporting of GHG emissions and air pollutants used to meet domestic and international reporting requirements, and enhancements to the SW system have resulted in a reduced burden on users. However, a need to continue expanding/adapting the system to accommodate new reporting requirements was identified.

Significant progress has also been made toward outcomes relating to the development of health- and environment-based ambient air quality standards and industrial emission standards, as well as final outcomes associated with lower emissions of air pollutants from regulated sectors and reduced ambient concentrations of PM_2.5 and ground level ozone.

Regulatory development linked to both air pollution and GHG emissions has taken place in collaboration with stakeholders, and the members of working groups view the consultative process as a strength. Opportunities were identified for improving the consultation process and clarifying the evidentiary base of the resulting regulatory instruments.

Regarding progress on outcomes relating to GHG emissions, GHG regulations have been developed for the transportation and electricity sectors, two of the largest emitting sectors, and these are coming into force. Regulatory development work has also occurred for the oil and gas, EITE and natural gas-fired electricity sectors, although these regulations have not reached the publication stage owing to a number of external factors, including changes to the government’s broader climate change policy. As a result, emissions reductions and progress toward the achievement of Canada’s Copenhagen target on GHG emissions have been limited. Concerns in this area were raised by the CESD in its Fall 2014 report, and EC, in response, has committed to enhanced efforts in this area.

Equivalency agreements with P/Ts are emerging as a useful and potentially widely-used tool for minimizing regulatory duplication and offering flexibility in achieving equivalent policy outcomes. The communication of a clear framework for their use will be an important consideration moving forward.

6.0 Recommendations and Management Response

The following recommendations are based on the findings and conclusions of the evaluation. As the senior departmental official responsible for CARA, the recommendations are directed to the ADM ESB, working with CARA partners from other branches within EC and HC, as appropriate.

Recommendation #1: Strengthen and formalize linkages between the regulatory and reporting elements of CARA to better harmonize requirements for new and existing regulatory reporting and inventories and improve resource planning for future enhancements.

The evaluation identified an opportunity to enhance the integration of regulatory and reporting activities, to better support harmonization with existing regulations and ensure that new regulations minimize duplication or undue reporting burden. Also identified was a need to clarify commitments to expand/adapt the single window system in order to accommodate new reporting requirements under CARA and other federal and provincial programs and avoid duplication, improve data quality and further streamline reporting.

Management Response to Recommendation 1

The ADM of ESB agrees with the recommendation.

Management Action

The Science and Risk Assessment Directorate (SRAD) in the Science and Technology Branch (STB) administers three important reporting tools that generate information relevant to CARA: the NPRI, the GHG Inventory and the “single window” reporting tool. The ADMs of ESB and of STB have established a forum to improve inter-branch coordination. Among other topics, this forum is addressing opportunities for CARA regulatory initiatives to leverage information generated under the SRAD reporting initiatives and to contribute to enhanced and streamlined reporting. To that end, ESB is adding a requirement to the regulatory Quality Management System (QMS) to require regulators and SRAD staff to meet at the earliest stages of the regulatory development process in order to discuss opportunities to use or modify the reporting mechanisms that SRAD administers to provide information that would otherwise be required in the regulation itself.

Recommendation #2: Develop and communicate a policy framework for equivalency agreements in order to facilitate clarity and consistency in their use, better understand future resourcing implications, and clarify expectations for provinces and territories.

Given that federal and provincial governments have a shared responsibility to protect the Canadian environment, CEPA 1999 includes the use of equivalency agreements as a tool for minimizing regulatory duplication and financial burdens, and offering flexibility in achieving equivalent policy outcomes. While one equivalency agreement has been finalized to date, it is expected that many more agreements will be requested by P/Ts over the next several years. There will be a need for clarity and consistency in developing equivalency agreements in order to optimize the potential benefits and support both efficient implementation and resource planning.

Management Response to Recommendation 2

The ADM of ESB agrees with the recommendation.

Management Action

ESB / Legislative and Regulatory Affairs Directorate (LRAD) is leading the establishment of a Community of Practice (CoP) in which ESB staff who have developed or have an interest in equivalency agreements will share experiences with their negotiation and development. The CoP aims to collect lessons learned as well as analyze similarities and differences experienced across agreements. The details of equivalency agreements vary significantly depending on the regulation and policies in place in the P/T partner concerned. CoP members will develop recommendations that LRAD will use to produce a policy framework on what equivalency agreements under CEPA should include and where divergence is to be expected.

Recommendation #3: Develop and share best practices / tools for stakeholder consultation to increase the efficiency and effectiveness of the regulatory development consultation process.

While recognizing that consultations must be flexible and adaptable to meet specific sector and regulatory development needs, the experience of the CARA federal partners in managing consultations is likely to yield some useful lessons that may be applied to improve the effectiveness and efficiency of consultations across the spectrum of CARA instruments and, more broadly, within the regulatory development process used by participating departments. While the federal consultative approach was generally identified by external stakeholders as a strength, opportunities for improvement were also identified. These included the timing for initiating the consultation process, requests to provide appropriate timelines for responding to analysis/ information, greater engagement by NGOs and the need for better articulation of the evidentiary base of regulations.

Management Response to Recommendation 3

The ADM of ESB agrees with the recommendation.

Management Action

Consultation and collaboration are important features of CARA. Recognizing the diversity of stakeholders involved in the development of CARA regulations, it is important that consultation processes remain flexible, allowing them to be tailored to the needs of particular stakeholders and subjects. ESB / LRAD will lead the development of a document compiling best practices for regulatory consultations, drawing on the experiences of CARA regulators and feedback from stakeholders.

Annex A: Program Logic Model

Annex B CARA Evaluation-Outcomes

For the purpose of the evaluation, the outcomes presented below were used to assess performance. They were developed based on program documentation, input received from scoping interviews with program managers, and feedback from the Horizontal Evaluation Steering Committee. These outcomes incorporate the concepts from the CARA logic model while more closely reflecting the outcomes associated with the 5 activity elements of CARA. They represent a combination of immediate and intermediate outcomes, in order to avoid repetition in data collection for related concepts. That is, they take into account that there is a continuum from immediate outcomes to longer term intermediate outcomes.

Final Outcomes (from the CARA logic model):

• Reduced emissions of GHGs and air pollutants from regulated sectors while maintaining competiveness in these sectors
• Reduced ambient concentrations of PM_2.5 and ground level ozone
• Canadians change behaviour to reduce exposure to air pollutants to improve their health

Annex C Supplementary Expenditure Tables

CARA Expenditures by Department and Branch, 2011-2014 (000's)

Environment Canada

Health Canada

National Research Council of Canada

CARA Expenditures by Expenditure Category, 2011-2014 (000's)

Annex D: Summary of Findings^Footnote72

Relevance

Performance - Efficiency and Economy

Performance - Effectiveness:

8. To what extent have CARA’s intended outcomes been achieved?

Unintended outcomes