Evaluation of the Habitat Conservation Partnerships Program: Appendix C
Evaluation strategy
Purpose and scope
The evaluation initially covered a five-year timeframe, from 2010-11 to 2014-15, but was updated with financial and performance data from 2015-16 and additional program documentation from 2016-17.
The evaluation focused on ECCC’s activities related to five of the seven components of this sub-program, both collectively and individually:
- Natural Areas Conservation Program (NACP)
- North American Waterfowl Management Plan (NAWMP)/North American Wetlands Conservation Council (NAWCC Canada)
- Wildlife Habitat Canada (WHC) Conservation Stamp Initiative
- Ecological Gifts Program (EGP)
- Canada’s participation in the Convention on Wetlands of International Importance (Ramsar Convention)
Expanding Family Oriented Conservation Programming and the National Wetland Conservation Fund (NWCF), the other two HCP Program components, were excluded from the scope of the evaluation. When this evaluation was conducted, they were still too new to allow for an assessment of their performance. An evaluation of the NWCF will be conducted in 2017-18.
Evaluation questions
Relevance
Continued need for the program:
- Is there a continued need for the program?
Alignment with government priorities:
- Is the program aligned with federal government priorities?
- Is the program aligned with Environment and Climate Change Canada’s strategic outcomes?
Consistency with federal roles and responsibilities:
- Is this an appropriate area for federal government involvement?
Performance
Achievement of expected results: assessment of progress towards expected results
- To what extent have results on land securement been achieved?
- To what extent have results on habitat management and stewardship been achieved?
- To what extent have results on participation and engagement been achieved?
Efficiency and economy: assessment of the degree to which the program is implemented in an efficient and economical manner
- To what extent is the cluster design appropriate?
- To what extent is the governance structure clear, appropriate and efficient?
- To what extent is the program delivery efficient and economical?
- To what extent is performance data collected?
Evaluation approach and methodology
For each evaluation question, the evaluation used multiple lines of evidence consisting of both quantitative and qualitative data collection methods described below. Multiple data sources provide information from various perspectives, which helps for triangulation and validation of evaluation findings, thus increasing their accuracy and credibility and strengthening the validity of conclusions. The following methodologies were used to collect evidence for the evaluation.
Document review
The evaluation team reviewed various types of documents, such as descriptive program documents, departmental and Government of Canada publications, relevant evaluations, reports, communications, research studies, performance and financial information and other internal planning and operational documents.
Key informant interviews
37 key informant interviews were conducted with 39 respondents:
- 11 ECCC program staff and senior management
- 28 external stakeholders (NGOs, government, board/council members, site managers and international stakeholders)
Online survey
After factoring in invalid email addresses, online survey questionnaires were sent to 871 individuals representing non-governmental organizations, government (federal, provincial, municipal), conservation authorities, private sector organizations, private individuals, universities and registered charities involved in one or more of the HCP Program components. 167 individual respondents completed the survey, for an overall response rate of 19.2%. The lower response rate (typical response rates are about 30% for this type of survey) may be due to the fact that respondents were only provided with two weeks to complete the questionnaire.
Many of the 167 respondents were familiar with more than one program component. As such, we received 276 completed program component responses.
The distribution of completed responses by program component is presented in Table 17.
| Completed responses by program component | Completed responses | Percent of total |
|---|---|---|
| Natural Areas Conservation Program | 45 | 16% |
| North American Waterfowl Management Plan/North American Wetlands Conservation Council (Canada) | 52 | 19% |
| Wildlife Habitat Canada Conservation Stamp Initiative | 80 | 29% |
| Ecological Gifts Program | 71 | 26% |
| Ramsar Convention | 28 | 10% |
| Total | 276 | 100% |
Case studies
To provide further insight, the evaluation team conducted six case studies (two projects from each of three program components: NAWMP/NAWCC (Canada), WHC Conservation Stamp Initiative and EGP). The NACP component was excluded from the case studies because information from an independent evaluation conducted in 2012 provided sufficient insight. Ramsar was also excluded from the case studies because of its lower materiality and risk. The six case study projects and their relevant timing are listed in Table 18.
| Program component | Case studies |
|---|---|
| North American Waterfowl Management Plan/North American Wetlands Conservation Council (Canada) |
|
| Wildlife Habitat Canada Conservation Stamp Initiative |
|
| Ecological Gifts Program |
|