Evaluation of the Community Ecosystem Partnerships Program: chapter 4

4.0 Findings

• 4.1 Relevance
• 4.2 Performance

This section presents the findings of this evaluation by evaluation issue (relevance and performance) and by the related evaluation questions. For each evaluation question, a rating is provided based on a judgment of the evaluation findings. The rating statements and their significance are outlined below in Table 2.A summary of ratings for the evaluation issues and questions is presented in Annex 1: Summary of Findings.

4.1 Relevance

Continued Need for Program

1. In the Okanagan-Similkameen PE, continuing rapid population growth and economic development in the region have led to significant threats to water availability and habitat loss thus demonstrating a need for careful management of the ecosystem to avoid further degradation and foster restoration.
   • Evidence from the document review demonstrates that the Okanagan-Similkameen is a unique region of Canada, recognized provincially and nationally as a ‘biodiversity hotspot’ and for the rarity of species (approximately 50 native species are listed as being at risk) and habitat. The mild continental and arid environment of the Okanagan-Similkameen makes it well suited for agriculture (ranching and vineyards) and attractive to developers. This combined interest has resulted in intensive urban and agricultural development, which in turn drives water demand and challenges supply. Development has resulted in significant habitat loss and degradation, with accompanying impacts to species.^Footnote13These factors suggest that careful management of the ecosystem is needed to avoid further degradation and foster restoration.
   • According to the Canada Water Act Water Availability Indicators (WAI),^Footnote14 the threat to water availability in the Okanagan Valley is rated “medium” (between 20% and 40%) based on the OECD classifications, indicating that poor water availability was a constraint on development, and significant investment was needed to provide adequate water supply to meet demand.
   • A 2009 EC study looking at water availability identified the Okanagan Valley as a high threat (meaning that more than 40%^Footnote15 of the water in rivers was withdrawn for human use).^Footnote16
   • A study on the effects of human population growth on the Okanagan River Valley recognized the Basin as one of Canada’s three most endangered natural systems due, in large part, to recent urban and agricultural development, dams, diversions and river channelization. The study concludes that the Okanagan has the highest ratio of population to water supply in Canada^Footnote17, current water consumption is not sustainable, and water quantities are too low to meet human and ecosystem needs.^Footnote18
   • Evidence from the document review is corroborated by findings from key informant interviews. Interviewees unanimously stated that there is a continued need for the CEP program in the Okanagan-Similkameen. Among the reasons cited were the pressures in Okanagan-Similkameen stemming from population growth in the region and accompanying increased demand on resources such as water and land (for farming, vineyards, roads), as well as the presence of a significant number of endangered species in the region that need protecting.
2. In the Atlantic Ecosystem Initiatives, the program needs to collect scientific data and research that focus on the entire Region, as issues related to water quality, watershed health and climate change which impact coastal areas transcend provincial boundaries.

• In the Atlantic ecozone (encompassing the four Atlantic Provinces) the intrinsic links between the coast, its biodiversity, settlement patterns, human use, and economy have resulted in increased unsustainable pressures (e.g., untreated or partially treated sewage discharge, eutrophication and red tides, increase in invasive species) on the health of the coastal ecosystem.^Footnote19 Key impacts on the health of the coast also result from land-based activities which originate up streams and rivers that then drain into the Atlantic Ocean. Finally, the impacts of climate change from rising sea levels, extreme weather and storm surge events are currently being felt and pose significant current and future risks to coastal areas.^Footnote20
• There was overwhelming support from all interviewees for the continued need of the AEI and its support of science and research.^Footnote21The main reasons cited for continued need for the program were: a) limited science capacity, baseline information, information sharing and knowledge of all existing data related to the Atlantic ecozone; b) a lack of compatibility of data and protocols; and c) limited long-term data on ecosystem health status and trends in the Region.
• Interviewees noted that in the absence of scientific data and research (related to such issues as water quality and quantity) generate through the AEI program, there would be a paucity of information on which the federal government could base its ecosystem management decisions in the Region, thus increasing the cost and time it takes to complete projects. For example, the AEI indirectly supports the Canadian Shellfish Sanitation Program at the local level through the work undertaken by eligible organizations that measure and monitor water quality. Others felt that there would also be disconnects between regional priorities and how they translate into national policy, as EC would have less information about what is going on in the regions.
• Although various other federal, provincial and private funding programs exist,^Footnote22 those funding programs have very specific sets of objectives (e.g., provincial funding programs bound by geography; DFO's funding program focuses on recreational fisheries), key informant interviewees suggested that a more holistic ecosystem-based approach is needed to address broader water issues in the Region, such as the restoration, enhancement and improvement of water quality and watersheds, monitoring and measuring water quality across the Region, or long-term planning through the development of comprehensive environmental management plans.

Alignment with Federal Priorities

The activities of the CEP program are consistent with federal and departmental priorities related to ecosystem health improvements and maintaining water quality and availability, and are aligned with Canada’s Federal Sustainable Development Strategy (FSDS).

• At the broadest level, activities under the CEP program contribute to “protecting the health and environment of Canadians”, which was identified as a key government priority in the 2011 Budget.
• One of the goals of Canada’s Federal Sustainable Development Strategy is maintaining water quality and availability. Specifically, the Government of Canada is committed to ensuring that everyone has access to a reliable and secure supply of clean water, and that water resources are used both economically and ecologically.^Footnote23 This includes a focus on cooperating on ecosystem initiatives, such as lake evaporation in the Okanagan ecosystem,^Footnote24 and investments to improve the understanding of the factors that influence water availability in EC’s West and North Region, such as funding to the OBWB for their Water Supply and Demand Study, a multiyear water resource assessment carried out in partnership by the OBWB and the BC Ministry of Environment.
• Using the ecosystem approach, the AEI is also an important mechanism to assist in achieving FSDS national goals and objectives within Atlantic Canada, specifically focusing on maintaining water quality and management in coastal ecosystems and adjacent watersheds.^Footnote25
• The Government of Canada’s Jobs Growth and Long-term Prosperity Economic Action Plan (Budget 2012) further confirms CEP’s alignment with federal government priorities. The document states “the Government is working together with partners to protect and restore Canada’s water resources for the benefit of all Canadians. Going forward, the Government will continue to pursue water quality and ecosystem health improvements in lakes and other bodies of water”.^Footnote26

Alignment with Federal Roles and Responsibilities

The activities of the CEP program are consistent with federal roles and responsibilities, including roles and responsibilities as outlined in the Department of the Environment Act and the Canada Water Act.

• Environment Canada’s CEP program is consistent with departmental roles and responsibilities as they are described in the Department of the Environment Act, which outlines the Department’s responsibility for the “coordination of the policies and programs of the Government of Canada respecting the preservation and enhancement of the quality of the natural environment.”^Footnote27
• EC is the federal agency responsible for the collection, interpretation and dissemination of standardized water quality data and information in Canada. EC also has a mandate for science and research on water quality issues, particularly as it relates to impacts on groundwater, sustainability, climate change and biodiversity. Both the AEI and Okanagan-Similkameen PE’s activities have focused on water issues, and their results are presented in the Canada Water Act annual reports.^Footnote28
• CEPprogram activities also focus on addressing water issues that are transboundary (inter-provincial and international) in nature, and there is therefore a need to involve the federal government, which can more effectively address these issues. The AEI encompasses ecosystems located in all four Atlantic Provinces, including the Bay of Fundy/Gulf of Maine (shared with the State of Maine). The Okanagan-Similkameen region is connected to the Columbia River system in Washington State.

4.2 Performance

Achievement of Intended Outcomes

1. Direct Outcome 1: Opportunity for Improvement
   Increased participation of governments and stakeholders in activities contributing to the goals and objectives identified in ecosystem-based management plans or to achieve ecosystem objectives
   
   Despite some progress, increased participation of governments and stakeholders continues to be an area needing improvement. In the Atlantic and Quebec Region, the program's contribution to this outcome is not as strong as in the past, due primarily to the closed funding process, as well as decreasing EC participation with community organizations. In the West and North Region, some progress has also been made to engage external organizations, though interviewees feel that the program has not been able to increase participation beyond a select group of regional organizations.
   
   
   • In 2012-2013,^Footnote29 AEI project activities resulted in environmental actions undertaken by 14 community groups and all 4 provinces (as funding contributors). Furthermore, funding led to over 320 workshops, sessions and events, and community volunteers donated some 14,000 hours to the accomplishment of 52 AEI projects. Without the existence of performance targets against which to compare the performance data collected, however, it is difficult to assess the adequacy of this progress.
   • One of the biggest impediments to greater participation is felt to be AEI’s current approach to funding only a limited number of prescribed organizations. A few interviewees noted that while there are other non-funded groups whose projects would be eligible, EC’s consistent funding of the same groups for the past 15 to 20 years has meant the funded groups no longer engage other stakeholders.
   • Funding recipients generally feel that it is a difficult process to get project volunteers, especially during an economic downturn. One funded recipient noted that, since the “window”^Footnote30 has disappeared, EC is not contributing to increasing project participation to the extent it had in the past. In addition, EC has stopped holding annual workshops with funded recipients to increase awareness of regional activities.
   • The Okanagan-Similkameen PE has made efforts to increase participation by working with different organizations and groups like the OBWB, the OCCP, and the Fraser Basin Council. Although the ability to increase participation is asked, but not required, of groups that receive funding, a few interviewees indicated that the program has not been able to increase participation beyond a select number of partners.
   • A few interviewees feel that, overall, the program needs to be more directly engaged with regional groups undertaking activities aligned with the program’s objectives, especially if the program aims to establish itself as the EC lead for ecosystem issues and knowledge in the Okanagan-Similkameen. As a result, these interviewees did not feel that the program contributed to increased participation in activities related to ecosystem health.
   • It should be noted, however, that the file review revealed that the Okanagan-Similkameen PE has been linked to a few of these regional groups.For example, CEP funding was provided in 2009-2010 to support the work of the South Okanagan-Similkameen Conservation Program (SOSCP)^Footnote31 and, since 2010-2011, the CEP has provided funding via the Allan Brooks Nature Centre for the activities undertaken by OCCP.^Footnote32 In addition, the CEP West and North Ecosystem Analyst sits on the Steering Committee for the OCCP, which provides opportunity for further engagement of key stakeholders also on the committee and identification of areas of potential further partnerships.
2. Direct Outcome 2: Acceptable
   Increased stakeholder capacity and knowledge
   
   The CEP program is contributing to increased stakeholder capacity and knowledge. In the Atlantic and Quebec Region, project activities have led to the sharing of knowledge and capacity through workshops and other events, as well as the sharing of tools developed through program funding across regional organizations. In the West and North Region, program activities have contributed to a better understanding of water availability issues in the Region through the Okanagan Water Supply and Demand Study and Lake Evaporation Study.
   • In 2012-2013, 15,400 AEI information products, such as pamphlets and booklets, were distributed to over 20,000 individuals. Furthermore, 5,600 individuals attended 320 training and information workshops, sessions and events on topics such as the impacts of climate change on coastal communities, environmental indicators, educational activities for students, geographic information management systems, water quality monitoring, updates of funded organizations' strategic management plans, and others. Funding also led to the creation of 74 jobs (including 22 student jobs).
   • AEI-funded projects also led to the development of knowledge and capacity that have been shared with other groups. For example, Saint Mary's University in Halifax is expanding the ability of community groups to do standardized water quality monitoring by providing a simple water quality monitoring kit that produces immediate results and teaching a standard protocol to ensure that results are accurate and comparable. According to interviewees, by expanding this idea to the four Atlantic Provinces, EC is now able to collect data in a consistent manner across the entire Atlantic Region.
   • In the Okanagan-Similkameen PE, interviewees feel that capacity and knowledge have increased to a moderate extent. For example, EC provided funding to support the development and execution of the Water Supply and Demand Study,^Footnote33 which aims to determine the inputs and outputs of water in the West and North Region. This study defines the parameters around which decisions could be made in the Okanagan Basin, and has drawn individuals from government, industry, and local community groups.
   • The Okanagan-Similkameen PE program is also collaborating with the OBWB on a lake evaporation study, which is presently underway. This had been identified by regional academia and local government partners as a key data gap in the accuracy of water prediction modeling processes.
3. Intermediate Outcome: Attention Required
   Coordinated ecosystem-based management
   
   Evidence shows that significant improvements are necessary in both regions in order to establish coordinated ecosystem-based management of environmental issues.
   • There was little evidence of coordination of activities found taking place within the department related to community ecosystem partnerships in the West and North and Atlantic and Quebec regions. Although a DG working group was established to inform DGs within the department of work taking place under sub-program 1.3.4, Ecosystem Initiatives, it has not met in at least two years due to changing Branch priorities while implementing Budget 2012 commitments. In the Atlantic and Quebec, an Inter-Branch Team was established to work collaboratively and support the implementation of an ecosystem approach in the region by sharing information, identifying gaps and opportunities, and providing guidance. Since the Budget 2012 reductions were implemented, however, the team no longer exists. In the West and North, there was no evidence provided of coordination activities being undertaken. Some interviewees were confused as to what role the program was expected to carry out in terms of coordination among regional partners.
   • Funding recipients feel that, prior to 2009-2010, the AEI did a better job of ensuring coordinated ecosystem-based management through strong participation in networking activities, such as bi-annual ACAP member meetings where groups would exchange information on projects. Recipients also have the sense that there was increased coordination across the Region, which allowed for more interactions, collaboration and sharing of experiences between the groups.
   • In 2005, EC convinced regional governments to take a more coordinated approach to regional development and to water management. This led to the development of the Okanagan Sustainable Water Strategy in 2008, a comprehensive guide to sustainable water management practices. As previously mentioned, however, the Okanagan-Similkameen PE is presently working with the OBWB, the regional body that governs water management in the Okanagan-Similkameen basin, on the Okanagan Water Supply and Demand Study.
4. Final Outcome: Opportunity for Improvement
   Beneficial uses and environmental quality of targeted ecosystems of federal interest are maintained or restored
   
   The evidence points to progress being made in the Atlantic and Quebec Region regarding the maintenance and restoration of beneficial uses and environmental quality, although it is too early to conclude whether this outcome will be achieved in the Okanagan-Similkameen PE.
   • According to AEI program staff and management, beneficial uses and environmental quality of targeted ecosystems in the Atlantic and Quebec Region have been maintained or restored to a moderate extent. For instance, the AEI 2012-2013 annual report indicates that 84 hectares of habitat were conserved (land and shoreline), 7,340 hectares were protected through special designation status, 718 landowners and resource users participated in habitat protection, and 70 kilometres of shoreline were preserved or restored (among other environmental results). As mentioned previously, without the existence of performance targets against which to compare the performance data collected, however, it is difficult to assess the adequacy of these results.
   • Staff and management also indicated that the program's contribution to this outcome could have been greater if more groups were able to apply for funding. In their view, it would be far better to work with the provinces and a larger pool of organizations in order to target other areas dealing with similar environmental issues in the Atlantic and Quebec Region.
   • With respect to Okanagan-Similkameen PE, it is too early to determine whether the program is making progress toward the final program outcome, although progress toward the final outcome may be in question given current resource constraints and challenges surrounding the achievement of immediate and intermediate outcomes.
5. External Factors
   • The file review of projects funded by both the AEI and Okanagan-Similkameen PErevealed that the most frequently noted external factors influencing the achievement of program outcomes were the following: insufficient project funding or funding that was expected from other project partners and that was not received (a factor in 10 projects), severe weather and natural landscape challenges (9 projects), project partners' capacity and competing priorities (7 projects), recipients' internal staff changes and departures (6 projects), and problems in soliciting community stakeholder and local volunteer participation (3 projects).
   • Additionally, the Okanagan-Similkameen PE has been able to build on existing governance structures (such as the Okanagan Basin Water Board, which was created by the three regional districts). Due to the Budget 2012 reductions, however, there was a period of six months where no one was working on program activities because of the focus on re-alignment.

While no unintended outcomes were reported for Okanagan-Similkameen PE, three such outcomes were identified for the AEI, including capacity building for young professionals, a sense of entitlement for the short list of approved funding recipients, and one Atlantic province decision not to fund organizations that already receive AEI funding.

• A few interviewees noted that one of the positive outcomes of the AEI is that it has drawn students and young people to local communities because there are opportunities for jobs. AEI funded groups have also become a good training ground for future young professionals, allowing them to learn and develop their skills and then take on management positions primarily within the public sector (either at the federal or provincial level).
• Interviewees also discussed a sense of entitlement on the part of some groups that make up the “ACAP Family,” whereby a select number of organizations that have received AEI funding continually over the years now feel that they are more entitled to funding than other groups and that they should be involved in how the program makes decisions. This is something that was facilitated by the design of the funding program.
• Furthermore, given that the program provides funding to a select number of community organizations, the province of PEI has decided not to fund the same groups who receive AEI funding, as they feel that there are other watershed areas that have similar environmental issues, and should therefore be able to access AEI funding. Interviewees noted that the province felt that this was unfair to other provincial community organizations and that these should also be eligible for AEI funding.

Although performance data is being collected and reported for the AEI, several weaknesses were noted, including a lack of performance targets, as well as indicators and activities not being clearly aligned with expected outcomes. In the Okanagan-Similkameen PE, no formal mechanisms for performance data collection and reporting exist presently, though there is a plan to develop a formal performance measurement strategy in the future.

• Although a CEP PAA-level program logic model has yet to be developed, the AEI has created an extensive list of indicators aligned to departmental sub-sub-programs in order to connect funded projects to the work being done in other areas of the department. Project proponents are required to choose a relevant set of indicators from this list to measure progress toward proposed activities, outputs and outcomes. This data is then compiled by the program and rolled up into an annual report, which also includes information on the number of projects, funding provided, and resources leveraged (in-kind and cash contributions from all sources).
• The document review and some key informant interviewees suggested a number of improvements to the program’s performance measurement, including: better alignment of the three categories of project indicators presented in the AEI annual reports (biodiversity and habitat; water quality; and sustainable ecosystems) with the three priority issues for AEI (biodiversity and habitat; water quality; and impacts of climate change); indicators that are more relevant to the activities that community groups undertake; improved clarity in terms of performance reporting expectations; and greater consistency in the manner different groups measure data in order to better support data aggregation across projects.
• The file review revealed that a vast majority of project proposals (20 out of 24, or approximately 83%) submitted by the organizations clearly outlined the expected outcomes of the projects, although it also showed that project reporting is done largely against activities and outputs rather than outcomes.
• Furthermore, not all AEI projects demonstrated good links between their planned activities, outputs and expected outcomes. In about half of the cases sampled, some of the outputs did not link plausibly to activities listed (e.g., outputs for some projects mention the number of restored habitats or kilometres of shoreline cleaned by volunteers even though project activities for these projects focused only on providing public awareness to the local population). As well, a few of the sampled projects have as many as a few dozen or more outputs, thus making it hard to link them plausibly to a much lower number of activities and outcomes. For other projects, outputs were not identified at all.
• In addition, the level of performance data collected and reported through these projects varies greatly, with about 15% of projects having none or only 1 or 2 performance indicators and approximately 38% of projects listing 30 or more indicators each. In several cases, as many as 70 and 80 indicators per project were established. Approximately 28% of sampled projects did not report on the established performance targets or the reported performance data fell significantly short of the established performance targets.
• Although a logic model for the Okanagan-Similkameen PE exists, there is no evidence that it is actively being used to collect performance data, as no formal performance measurement mechanisms are in place. Program staff noted that they are planning to work on a performance measurement strategy to better track the impact of program activities and inform decisions on where and how to invest program resources.
• The file review revealed that a strong plausible link between activities and outcomes was found in half of the 8 project files reviewed. In the other half, no project-specific intended outcomes were outlined, but the activities were still aligned to one or more of the expected outcomes of the contribution program “Contributions to Support Sustainable Ecosystems” under which the contributions are administered. Outcomes were not reported on and, unlike the AEI project proposals, they were also not clearly outlined in project proposals. Of the 8 projects, only one mentioned intended outcomes, while the other 7 only indicated activities and outputs.

Extent to which Performance Data Informs EC’s Decision-Making Processes

• With respect to the AEI, performance information is being made available to inform decision-making. Generally, the information received from individual funding recipients is compiled and presented in an annual report, which is then shared with senior management. The annual report presents some of the key program results and accomplishments, providing senior management with information on the extent to which the program is achieving its overall objectives.
• For the Okanagan-Similkameen PE, no established data collection and reporting structure presently exists. Senior management indicated that the program does not have the capacity to capture a wide range of performance data on the Okanagan-Similkameen PE, although there is a plan to develop a formal performance measurement strategy. Presently, performance data is collected at the project-level from funding recipients and used to inform decisions regarding resource allocations for the following year.

Demonstration of Efficiency and Economy

The evaluation determined that, in general, the CEP program was delivering activities and outputs at a low cost. Evidence does point to some potential improvements related to such areas as better communications of program objectives and priorities, and improving collaboration with funded recipients. Funded recipients in both regions have generally been successful at leveraging contributions from other sources, especially for the AEI.

• A common indicator to assess the operational efficiency of contribution programs is the administrative ratio, which compares operational costs (salaries and O&M) to G&C funding disbursed. The administrative ratios for CEP Atlantic and Quebec and West and North are 0.078 and 0.062, respectively. These ratios are much lower than other EC G&C programs, such as the Lake Winnipeg Basin Stewardship Fund, which has a ratio of 0.22, and the Lake Simcoe Clean-up Fund, which had a ratio of 0.15 as of 2011-2012. These latter G&C programs, however, have an open competitive process to disburse G&C funding, and so would be expected to have a higher administrative ratio.
• With respect to AEI, interviewees offered a number of reasons why they generally feel that the program is being delivered efficiently, including:
  • the co-location of EcoAction, EDF and AEI staff, which facilitates communication between funding programs and improves delivery of all three (e.g., reducing duplication through sharing or referring funding proposals among programs);
  • combining offices in the Atlantic and Quebec regions, which has resulted in experienced staff in an established ecosystem-based program (St. Lawrence Action Plan) sharing their experiences and lessons learned with AEI staff;
  • funding the same groups each year through a closed funding process,^Footnote34thus ensuring processes are streamlined and funding recipients are very familiar with overall program objectives, priorities and processes; and
  • various other operational characteristics, such as the phased approach to proposal submissions to the Minister's Office, an increased use of online applications, good relationship between EC program managers and traditionally funded groups, and partnerships with universities, which help to keep costs low (e.g., use of labs, field sampling equipment).
• Despite efficiencies from co-location, senior management indicated that operational expenditures were still too high in comparison to other programs and so the total number of FTEs to deliver the contribution program was reduced from 2.3 to 1.3 in late 2013.
• To improve the AEI's efficiency, funding recipients also identified a need to better communicate program changes and priorities (e.g., the move from ACAP to AEI was not felt to have been adequately communicated to partners and eligible organizations) and to better collaborate with funded groups through in-person interaction and annual workshops in order to share experiences and lessons learned.
• In the Okanagan, one of the key program efficiencies identified by interviewees concerned partnerships with local governments and regional organizations, such as the OBWB, to participate in discussions and collaborate on projects to address environmental issues in the Region through established governance structures. For example, as part of the Lake Evaporation Study led by the OBWB, the Okanagan-Similkameen PEprovided funding for equipment (large yellow buoys and two land-based stations), thus overcoming a significant hurdle for that particular project.
• Interviewees also indicated a few possible improvements to program efficiency, including:
  • modifying the proposal solicitation structure for the contribution program to expand eligibility, increase communication and collaboration between program staff and funded groups, and reduce proposal approval times; and
  • establishing clear performance measurement requirements in order for funding recipients to collect relevant performance data to demonstrate project results.

Leveraging

• Evidence from the administrative file review suggests funding recipients have achieved considerable success in leveraging cash and in-kind contributions from multiple sources, averaging 9 contributors per funding agreement. In both regions, funding partners most often include municipal and provincial governments, non-governmental organizations, other federal government departments, industry and private businesses, academia, and, in AEI programs, local volunteers and youth. For most funded recipients, leveraging most often involves in-kind contributions because recipients depend on the knowledge, equipment and volunteers in the community.
• Since 2008-2009, EC has provided approximately $6.67 million to 287 projects in the Atlantic Provinces and leveraged an overall value of $20.81 million (cash and in-kind). Thus, funded recipients leverage an average of more than $3 ($3.12) for every dollar of EC funding, for a ratio of 3:1.The largest non-federal partners are provincial governments (average of 16% of total project value annually) and non-governmental organizations (14%).
• As mentioned previously, however, in the view of a few interviewees, the closed approach to funding has meant less engagement of other stakeholders and a resulting decrease in the amount of funding coming from these other sources.
• In the Okanagan-Similkameen, for every dollar of EC funding, the program leveraged an average of just under $2 ($1.86), for a ratio of approximately 2:1.There were a few instances, however, where EC was the sole funding source.

The evaluation found that overall governance mechanisms were clear and effective. With respect to AEI, roles and responsibilities were generally clear and commonly understood. In the case of the Okanagan-Similkameen PE, program objectives and the roles and responsibilities of the program were not clearly communicated and commonly understood. Furthermore, the perception is that the program has not adequately engaged other directorates to ensure that a coordinated departmental ecosystem-based approach is implemented in the region.

• Overall, most interviewees reported that governance mechanisms in both regions are clear and effective, with a clear and commonly understood reporting structure. In the Okanagan-Similkameen PE, an EC-specific governance structure was never created, as the program relies instead on existing local bodies (such as the OBWB) and local community groups to coordinate activities with partners and stakeholders.
• Under the current structure or organization of activities in the region, no evidence was found of any mechanism to coordinate multiple departmental activities occurring in the same priority ecosystem. If those areas happen to be a priority ecosystem, one senior manager suggested that there needed to be some way to tie what EC is doing in terms of science, regulatory measures, monitoring, etc. to what local and regional jurisdictions are doing. Key informants feel that the Okanagan-Similkameen PE in particular could be doing a better job at engaging other directorates across EC and ensuring that departmental activities are carried out in a more cohesive manner. It is worth noting that the RDG W&N outlined plans and initial steps to establish a governance structure to coordinate EC intervention actions in the Okanagan. A business plan was developed for the Okanagan-Similkameen PE and initial meetings of an internal coordination group were held. However, the business plan and associated resources were never formally approved, as Budget 2012 implementation took priority over approving new initiatives seeking additional resources.
• One senior manager noted that, when priority ecosystems are designated, two governance structure considerations should be addressed. First, when previously individual program lines are brought together under one priority ecosystem, a coordinated approach needs to be established so that the ecosystem works as a cohesive whole. Second, EC activities in areas such as science, regulatory measures and monitoring in the ecosystem should be tied to provincial and local governments' activities in that ecosystem, since decisions that affect the environment are increasingly being made by those levels of government.
• Interviewees generally feel that the management of day-to-day operations is clear and commonly understood. Despite some confusion among program staff in terms of senior management approval processes following changes to the departmental governance structure in 2012, these issues have since been resolved.
• Clarity of Roles/Responsibilities:
  Most program staff and external stakeholders interviewed saw their roles as clearly defined and understood. With respect to the Okanagan-Similkameen PE, internal interviewees, however, feel they do not have a clear understanding of the program objectives and the role of the program in the region. Program representatives noted that discussions are underway to clarify the roles and responsibilities of departmental groups involved, particularly in the areas of program policy and priority-setting, outreach and solicitation of proposals, proposal assessment and notification of decisions, and ongoing management of awarded contribution agreements.
• Clarity of Priorities:
  Most AEI interviewees said that priorities are well communicated and understood and funding recipients noted that the application process clearly communicates program objectives and priorities. In the Okanagan-Similkameen PE, however, interviewees were of the opinion that, while priorities are generally understood by program personnel, other stakeholders (including funding recipients and partners within the department) tend to be confused about the program's overall objectives and role in the region, including a lack of standard application materials.
• Approval Processes:
  A majority of interviewees indicated that the approval processes are clear and effective. There are a significant number of interviewees who feel that the approval process has improved over the last few years. There was also evidence that showed that the new phased in (batch process) in the AEI has increased efficiency. Although a few funding recipients mentioned that the timelines for submission and approval are too late in the season, most feel that they are receiving their funding at a more appropriate time in the year than was previously the case. In the Okanagan-Similkameen PE, a few interviewees feel the approval processes are effective but not always clear. Interviewees indicated that this was a bi-product of the program's greater focus on partnership building and stakeholder coordination than on funding activities.

Under the AEI, the closed nature of the funding process is seen as a major impediment to the achievement of program objectives. The program, however, is in the process of transforming its program delivery model to include open funding.

• Presently, to receive AEI funding, a recipient must be one of a limited list of eligible organizations. Many interviewees noted that an open process, accepting applications from all organizations that may apply, would allow the AEI program to more effectively achieve its intended outcomes and overall objectives through better proposals and improved partnerships and knowledge sharing. An example of such a competitive application process is the EcoAction program. As one interviewee noted, “competition drives innovation, which drives good development, new ideas and ways of doing things.”
• Key informants report that the program is currently considering changing existing delivery processes, which could include moving from a closed to competitive funding process starting in 2015-2016.Although the federal-provincial MOU with the Atlantic Provinces on Environmental Cooperation, which ended in June 2013, is not expected to be renewed, AEI is expected to collaborate with the provinces to gain their support and ensure the provinces are involved in any future AEI approach. This approach would be expected to better align AEI objectives with government-wide priorities, while strengthening federal-provincial relations.

In the West and North Region, despite resource reductions, program objectives and activities have remained the same.

• Since the Budget 2012 reductions were introduced, the number of FTEs carrying out activities related to community ecosystem partnerships in the Region^Footnote35 decreased from 3.4 FTEs in 2011-2012 to 0.9 FTEs in 2012-2013 and the total budget was reduced from $715,980 to $475,388 in the same years. The evaluation, however, found no evidence of corresponding changes to program objectives or program activities following the significant reduction in resources. Senior management reported that the overall objectives of the Okanagan-Similkameen PE have not changed even though the program continues to play an appropriate role and maintains a liaison with local bodies like the OBWB (including the Okanagan Water Stewardship Council).