Formative Evaluation: Enabling Fund for Official Language Minority Communities and the Related Governance Structure, January 2010

Executive Summary

The Enabling Fund for Official Language Minority Communities was established in 2005 with a program budget of $36 million spread over 3 years (2005-2008). It replaced the Support Fund for Official Language Minority Communities, which was first announced in 1999 for a three-year period and then renewed until 2005.

The objective of the Enabling Fund is to enhance the development and vitality of official language minority communities by strengthening capacity in the areas of human resource development and community economic development and by promoting partnerships at all levels, especially with federal partners. The Fund plays the role of facilitator by helping official language minority communities to increase and strengthen partnerships, and to gain access to additional sources of funding. The Fund's budget is allocated under contribution agreements with 14 eligible organizations.

Evaluation Limitation

The formative evaluation methodology used two qualitative data collection methods only. The use of quantitative data collection methods was not possible because the Enabling Fund program, through contribution agreement signatories, does not provide programs and services directly to individuals or employers. In addition, the list of community partners involved in community projects or activities with the RDÉEs and CEDECs is not systematically collected and therefore was not available.

Evaluation Findings

2. Evidence of Progress Towards Achievement of Short-and Medium-term Objectives

Immediate Outcome No. 1: Increased Capacity and Accountability of HRSDC and Service Canada

A protocol signed by HRSDC and Service Canada identifies their responsibilities along with some additional details regarding the process of cooperation and communication at the managerial level. HRSDC is responsible for developing policies and programs for the Enabling Fund, including the development of horizontal policies. Service Canada is responsible for operating the Enabling Fund, including the approval and management of the contribution agreements.

The document review showed that there is considerable room to improve communication and joint support between HRSDC and Service Canada. For example, the designated directors general did not meet each quarter, as provided for in the protocol.

Key informants felt that HRSDC's logistical support is very good, but that HRSDC should offer better strategic support and improve the operations of the national committees (e.g. by strengthening strategic support for the committees). Signatories to the contribution agreements appreciated the commitment of Service Canada officials to providing operational leadership, but would like to meet with them more often in the field. As well, HRSDC managers felt that the Department should improve its data processing capabilities and capacity for information sharing on program results.

Immediate Outcome No. 2: Increased Capacity of the Signatories to Act as an Agent for Human Resource Development and Economic Community Development

Key informants and the document review confirmed that the signatories have strategic and operational plans. The document review also showed that activities conducted by the signatories include forums, community consultation sessions, community plans and projects, and support for community organizations. The signatories act as agents of development through their plans and activities.

As noted earlier, the monthly reports do not provide information on expected or actual impacts in the field. Therefore the evaluation was not able to provide a summary of the expected or actual impacts of the funded activities at the local level. This also creates a limitation to ongoing program monitoring.

Intermediate Outcome No. 1: Increased Capacity of the RDÉEs and CEDECs to Develop or Assist in the Development of Community Plans and Projects

The activities of the RDÉEs and CEDECs appear to extend to all official language communities. In addition, key informants indicated that the RDÉEs and CEDECs have appropriate planning tools and expertise such as professional development workshops, toolkits and resource inventories.

Key informants indicated that the RDÉEs and CEDECs are called upon (both formally and informally) by corporations, community organizations, and non-profit organizations to help in drawing up strategic plans and business plans. As well, key informants in the field provided examples of RDÉEs and CEDEC leadership supporting the mobilization of community resources (e.g. the development of numerous community projects, and assistance with social enterprise building).

Immediate Outcome No. 3: Increased Knowledge Within the National Committees, the RDÉEs, CEDECs and Federal Departments and Agencies of the Issues Faced by Official Language Minority Communities

Evidence from the document review and key informants indicated that the management tools and research reports enhance awareness of community issues. For example, the document review indicated that Appendix A of the contribution agreement clearly sets out the problem areas and the interpretations that underlie the targeted activities. In addition, the RDÉEs and CEDECs consult on a regular basis with the communities involved (e.g. through forums, surveys, meetings and field assessments of the relevance of the projects).

Immediate Outcome No. 4: Improved Access to Existing Federal Programs for Official Language Minority Communities

Key informants from HRSDC and Service Canada indicated that no well-defined horizontal process has been developed to help improve signatories' access to existing federal programs. In the absence of an overall process, the representatives of federal agencies and departments sitting on the national committees take on an information and advisory role in communicating information on their programs to the community representatives. As well, an inventory of existing programs was developed by Canadian Heritage and made available on the department's Web site.

The document review revealed that the RDÉEs and the CEDECs signed a variety of other contribution agreements with federal agencies and departments in 2006-2007: 98 for funding, and 91 involving in-kind contributions. The data on these other contribution agreements were drawn from the monthly reports on partnerships. The available data also showed that total contributions by partners of the RDÉEs and CEDECs amounted to nearly $25 million, including just over $7 million from federal institutions. The data also showed considerable regional variation in the amounts. ^{Footnote 4}

Five CEDECs (out of eleven) and eight RDÉEs (out of twelve) felt they had better knowledge of federal programs. Most stated they had better access to federal programs, but they felt this was due to their own efforts and not due to the established information sharing mechanisms.

Intermediate Outcome 2: Increased Mobilization of Resources and Investments from Public and Private Sectors and Consolidated Financial Partnerships

The available documentation showed that the CEDECs and RDÉEs obtained the equivalent of $19.5 million in financial contributions and $5.3 million in-kind contributions. The document review also indicated that non-federal contributions (public, private, provincial and others) accounted for $17.6 million of the overall total of $24.8 million. The evaluation was not able to determine what portion of these contributions can be attributed to the Enabling Fund (i.e., would not have occurred in the absence of the program).

3. Horizontality

The Enabling Fund is designed to ensure horizontality by funding the coordination of the two national committees (which bring together community representatives and representatives from various federal agencies and departments). In addition, the inclusive structure of the RDÉEs and the CEDECs ensures community representation.

The leveraging effects in terms of partnerships and investments (discussed above) represent tangible results with the involvement of partners from the private, public, municipal and community sectors. As well, federal officials indicated that the horizontality encouraged by the Enabling Fund has helped communities to get their message across. At the same time, however, key informants provided a number of suggestions for improving the effectiveness of the program design features related to horizontality. Their suggestions included:

• increase the strategic content of the national committees;
• improve the operations of the national committees, for example by placing greater emphasis on concrete results and follow-up; and
• consider adopting a more inclusive and participatory approach involving other community representatives.

Recommendations

Based on the results of the formative evaluation, the following recommendations for follow-up action are set out.

Greater effort and resources should be devoted to:

• Building and maintaining the internal capacity of Contributions Agreements Signatories and the CEDECs. Particular emphasis is required on:
  • addressing issues relating to staff retention, succession planning and decreasing reliance on support received from volunteers (in the case of the CEDECs particularly); and
  • securing additional funding from diverse sources (Federal, provincial and private sector) to support organizational activities.
• Improving the data collection and accountability processes. It is recommended that standardized data collection templates be developed and used by Contribution Agreement Signatories to support the accurate capture of program results and monitoring information into an electronic database at HRSDC. Such an electronic database will serve as a pillar for reporting, monitoring and evaluation activities.
• Improving the operations of the national committees by:
  • increasing the strategic content of discussions at the national committee meetings and by placing greater emphasis on achieving concrete results, including follow-up; and
  • adopting a more inclusive and participatory approach involving other community partners that reflect particularly regional diversity.

Management Response

1. Main Results

The Evaluation Report is generally favourable, pointing out that the Enabling Fund represents an improvement over the previous Support Fund. The Evaluation Report notes that the objectives of the Enabling Fund are clearly defined and well understood by agreement signatories. The signatories operate satisfactorily as agents of development. Their leadership is recognized and seen as essential to several projects, and they contribute to improving the social and economic structure of OLMCs. Despite the need to review and improve the quality of reports, approval and reporting processes have been strengthened and now include regular production of formalized reports. The Evaluation Report also notes that the Enabling Fund generates a substantial leveraging effect in terms of investment by the federal, provincial and territorial public sectors and by the private and non-profit sectors, in the form of financial or in-kind agreements. The Enabling Fund ensures horizontality by funding the coordination of the two National Committees. They bring together community representatives and representatives from various federal agencies and departments and constitute a preferred means for consulting communities.

The Formative Evaluation Report sets out three groups of recommendations. The next section presents these recommendations and proposed corrective actions.

2. Recommendations Requiring Follow-Up and Corrective Action

2.2 Improving the data collection and accountability processes

Improving the data collection and accountability processes. It is recommended that standardized data collection templates be developed and used by Contribution Agreement Signatories to support the accurate capture of program results and monitoring information into an electronic database at HRSDC. Such an electronic database will serve as a pillar for reporting, monitoring and evaluation activities.

The Department agrees with the recommendation to improve data collection in order to strengthen performance measurement. The Department acknowledges that data collection and performance measurement are a particular challenge for the Enabling Fund. The mechanisms in place to ensure accountability and show results lacked depth and did not represent a uniform approach to measuring results for the communities. The Department has worked with agreement signatories to address these gaps in recent months and will continue to do so.

The Department worked with agreement signatories to develop a new Performance Measurement Framework which is aligned with the commitments and directives set out in the Roadmap. As a result, it reviewed the program's logic model and developed a set of 20 performance measurement indicators and validated them with the agreement signatories. Additionally, as follow-up to these discussions, the Department will implement a model of community maturity and signatory capacity (currently being developed) that will measure the vitality of the communities in terms of economic and HR development. In the coming years, the program will have the opportunity to collect new quantitative data that will strengthen the measurement of direct results for OLMCs and the program's overall performance.

The Department is working on improving data collection and results sharing. It has developed an administrative database prototype which will be improved to reflect the new Performance Measurement Framework and to ensure it is an effective and homogeneous data collection tool. The Department has begun working on aligning the prototype's design with the revised logic model. The Department will also use alignment with the results-based management accountability framework, risk-based audits and routine formative evaluations to ensure that the Enabling Fund produces concrete results for OLMCs and can report on those results more easily. The sharing of results, success stories, and performance measurement reports will assist in showing the progress made by the program. The Department has also strengthened its research capacity with regards to OLMCs in the recent months. It is pursuing a number of research projects that will improve its capacity to strengthen the program.

2.3 Improving the operations of the National Committees

Improving the operations of the National Committees by:

• increasing the strategic content of discussions at the National Committee meetings and by placing greater emphasis on achieving concrete results, including follow-up; and
• adopting a more inclusive and participatory approach involving other community partners that reflect particularly regional diversity.

a) Increasing the strategic content of discussions at the National Committee meetings and placing greater emphasis on achieving concrete results

The Department agrees with the recommendation and proposes several measures to increase the strategic content of discussions at National Committee meetings and to place greater emphasis on achieving concrete results.

To respond, the Department will engage the community sector and its federal partners in developing a strategic action plan for the National Committees, with achieving concrete results as the objective. Preliminary discussions of this issue have already been held with the various partners, and will be expanded at upcoming National Committee meetings. The recent creation of issue-specific working groups is evidence of the parties' commitment to increasing the strategic content of National Committee discussions. In addition, the Department has resumed its responsibility of co-chair to the National Committees since 2008. This will tie its leadership to the Committees' work and provide them with better strategic support. Also, the inclusion of the National Committees in the Enabling Fund's logic model will help better define the role of the National Committees Federal Secretariat and promote more strategic discussions.

b) Adopting a more inclusive and participatory approach involving other community partners that reflect particularly regional diversity

The Department agrees with the recommendations and will explore the possibility of adopting a more inclusive and participatory approach within the National Committees, to increase the involvement of community partners active in community economic and human resource development among OLMCs.

The Enabling Fund's horizontality would be strengthened if the National Committees adopted a more inclusive approach.

To respond, the Department will engage community stakeholders with the objective of broadening community representation to the National Committees. These stakeholders will have to strengthen regional diversity and the diversity of groups interested in community economic and human resource development. In addition, over the years, the Department worked to increase the diversity of government stakeholders who sit on the National Committees by inviting regional economic development agencies and other federal departments and agencies. The Department will continue to ensure other departments with related sectoral mandates are included in National Committee discussions. To improve the strategic content of National Committee discussions, ensure the achievement of concrete results, and enable a more inclusive participation by community partners, the Department will also work with community stakeholders to better define the roles and mandates of the National Committees and clarify participants' expectations.

3. Conclusion

The Formative Evaluation has provided helpful insight and recommendations to strengthen the design and delivery of the Enabling Fund. The Department has already used the findings of the Formative Evaluation to help realign the program and make it more efficient. It will continue to do so in the future. Accordingly, the Department will continue working with signatories and other key stakeholders to achieve the objectives established under the Enabling Fund for OLMCs program.

1. Introduction

The formative evaluation of the Enabling Fund for Official Language Minority Communities focuses on the design, implementation and governance of the Fund. It also focuses on examining the achievement of the program's short- and medium-term objectives. As well, it examines horizontality ^{Footnote 6} issues and follows up on the findings and conclusions of the formative evaluation of the Enabling Fund's predecessor program, the Support Fund for Official Language Minority Communities. ^{Footnote 7}

1.2 Objectives of the Enabling Fund

The Enabling Fund's objective ^{Footnote 8} is to "enhance the development and vitality of official language minority communities by strengthening capacity in the areas of human-resource development and community economic development and by promoting partnerships at all levels, especially with federal partners."

The activities of the Enabling Fund "do not represent the only solution to the problems faced by these communities, but they will help create essential synergies. The Enabling Fund plays the role of facilitator by helping official language minority communities establish other partnerships, gain access to other sources of funding and/or strengthen existing partnerships."

To achieve the objective of the Enabling Fund, Human Resources and Skills Development Canada (HRSDC), Service Canada and their partners carry out a number of activities. These activities involve the development of administrative procedures relating to the approval process for applications for funding, research, analyses and communications. These activities are to support the achievement of specific results over the short, medium and longer terms. The intended outcomes of the Enabling Fund are shown in Table 1.

Table 1: Enabling Fund's Immediate, Intermediate, Longer-term and Final Outcomes

Immediate outcomes (1 year)

i) Increased internal capacity and administrative accountability for the Enabling Fund at Service Canada and HRSDC;
ii) Increased capacity of signatories to contribution agreements to intervene as agents of human resource development and community development;
iii) Increased knowledge of the issues faced by official language minority communities within the national committees, the Réseau de développement économique et d'employabilité (RDÉEs), the Community Economic Development and Employability Committees (CEDECs), federal departments and agencies, and other partners;
iv) Improved access to existing federal programs for official language minority communities.

Intermediate outcomes (2-3 years)

i) Increased capacity of RDÉEs and CEDECs to develop or assist in the development of community plans and projects (geographically, provincially, or sectorally);
ii) Increased mobilization of resources and investments from public and private sectors and consolidated financial partnerships;
iii) Improved integration of issues of human-resource development and economic development of official language minority communities in designing policies and programs at HRSDC.

Longer-term outcomes (3-5 years)

i) Increased relevance of community plans and projects to actual needs and implementation at the local, provincial or territorial and sectoral level;
ii) Consolidated partnerships in all federal institutions and with community stakeholders.

Final outcome (timeline greater than 5 years)

Strengthen the development of human resources and community economic development in official language minority communities.

1.3 Governance Structure

HRSDC is responsible for developing policies and programs relating to the Enabling Fund. Service Canada is responsible for delivering the Enabling Fund.

The Fund is delivered in the form of contribution agreements. Access to the Enabling Fund is limited to the following fourteen eligible organizations:

• Réseau de développement économique et d'employabilité RDÉEs Canada, representing the community table of the National Economic Development and Employability Committee (national Francophone committee);
• the National Human Resources Development Committee acting on behalf of the minority Anglophone community (national Anglophone committee); and
• twelve regional RDÉEs.

The Government of Canada signed a memorandum of understanding with each national committee to establish a framework for cooperation to foster the vitality of official language minority communities in Canada and to support their economic development and strengthen the capacities of the communities involved.

Figure 1 provides an overview of the governance structure of the Enabling Fund:

• Thirteen departments and agencies signed the memorandum of understanding on behalf of the Government of Canada with the national Francophone committee.
• Nine departments and agencies signed the memorandum of understanding on behalf of the Government of Canada with the national Anglophone committee.
• The national Francophone committee and national Anglophone committee were each mandated to provide advice to assist in directing the policies, programs and services of HRSDC, Service Canada and the other federal partners. These two national committees also carry out a number of essential functions in the areas of coordination, information, liaison, research and development.
• The two national committees include representatives of federal departments and agencies (government table) as well as representatives from the official language minority communities (community table). Each of the national committees is co-chaired by a representative of the federal government and a member of the community involved.
• At the provincial or territorial level, each RDÉE has a delegated organization tasked with the implementation of the contribution agreements signed under the framework of the Enabling Fund, except for Ontario. ^{Footnote 9} For example, the agency delegated to implement the RDÉE in British Columbia is the British Columbia Economic Development Corporation.
• The CEDECs receive funding from the national Anglophone committee and do not sign contribution agreements under the Enabling Fund.

1.4 Financial Resources

As noted above, the Enabling Fund was established in March 2005 with a program budget of $36 million spread over three years. The budget is allocated under contribution agreements with 14 eligible organizations. Table 2 shows the budget distribution for 2006-2007 and 2007-2008.

• Source from the above table Source: HRSDC/Service Canada

2. Formative Evaluation Methodology

This formative evaluation of the Enabling Fund covers the period from 2005 to 2008. The evaluation methodology is based on two qualitative data collection methods: a document review, and key informant interviews.

2.2 Interviews

Face-to-face interviews were conducted with 23 key informants in the National Capital Region. As well, 86 persons across Canada were interviewed by telephone. The interviews were conducted in the respondent's language of choice.

The face-to-face interviews were conducted with the following federal officials:

• Service Canada program managers and officers (7);
• HRSDC program managers and officers (5);
• federal departments and agencies sitting on the national committees (8); and
• representatives of federal departments involved with the official languages file (3).

The telephone interviews were conducted with the following persons in the field:

• the directors of signatories to the contribution agreements, including the RDÉE s, RDÉE Canada, the community table of the national Anglophone committee and the CEDECs (25);
• the community partners of the RDÉE s and CEDECs (59); and
• the Quebec Community Groups Network (QCGN) and the Fédération des communautés francophones et acadienne (FCFA) (2).

To make the collected information easier to compile and analyze, the results of the interviews were incorporated into an evidence matrix based on the evaluation issues and questions.

2.3 Evaluation Limitations

The formative evaluation methodology used two qualitative data collection methods only. The use of quantitative data collection methods was not possible because the Enabling Fund program, through contribution agreement signatories, does not provide programs and services directly to individuals or employers. In addition, the list of community partners who are involved in community projects or activities with the of RDÉE s and CEDECs is not systematically collected and was not available.

3. Findings Arising Out of the Evaluation

3.1.1 Enabling Fund Versus Support Fund

Evaluation Question: In terms of design, how does the Enabling Fund differ from the Support Fund?

The key informants state that the Enabling Fund's approach puts more emphasis on community economic development by broadening the original scope of the Support Fund. Therefore, the Enabling Fund is better suited to the specific needs of the clientele.

The key informants also felt that the Enabling Fund offers potential leveraging effects on community vitality, and on employment and economic development, through partnership agreements with the private, community and government (federal/provincial/municipal) sectors. As well, they feel that the continued emphasis on leveraging, which is a basic feature of the program, strengthens horizontality and contributes to the program's consistency with the broad government priorities affecting official language minority communities.

In addition, the key informants state that the Enabling Fund includes the following operational improvements over the Support Fund:

• the Enabling Fund has a more structured approval process (based on pre-established criteria for funding applicants);
• the contribution agreements were signed for two years for most of the eligible organizations in 2006-2008 (compared to one year under the Support Fund);
• the Enabling Fund broadens the framework of allowable expenses to community capacity building and community economic development activities; and
• accountability is increased through the regular production of formal reports.

With base funding to support a development infrastructure, the RDÉE s and CEDECs have consolidated their structure and networks.

The document review corroborates the findings of the key informant analysis by showing that the Enabling Fund is more closely tailored to the specific nature of its target population than its predecessor program. As well, the documents analyzed show evidence of an intention to strengthen:

• the relationship between the program's mandate, objectives and activities;
• the relevance of the partnerships;
• long-term funding;
• communications; and
• the accountability process.

Basis and Rationale for Changes Made to the Program

Evaluation Question: On what basis were the changes made to the program?

The formative evaluation of the predecessor program (the Support Fund) highlighted the need for greater coherence within the government, greater flexibility in dealing with partners, simpler processes and improved accountability.

Consultative processes with signatories to the contribution agreements were carried out in 2003 and 2004. These consultations took the form of workshops involving official language minority communities and HRSDC representatives. The purpose of the workshops was to define community priorities and the program's management requirements. Further consultations have been conducted to develop management tools for the Enabling Fund (as discussed in Section 3.1.2 and Section 3.1.5).

Both the document review and the key informant analysis indicate that changes made to the program were guided by several key factors:

• The need for more horizontality was identified as an important factor influencing changes made to the program. To better respond to the needs of the communities, there was a need to establish a link with the entire range of federal programs in order to optimize synergies and to avoid the duplication of effort and the overlapping of funding.
• Another important factor was the need for more stable funding to ensure continuity in community actions.

Funding Application Approval Process

Evaluation Questions: What is the current approval process for funding applications? What adjustments have been made to the process since the 2002-2003 (formative) evaluation (of the predecessor program)?

As noted in Section 1.3, the request for proposals process is limited to a fixed number of organizations, and is managed according to a predetermined schedule. RDÉE Canada, the community table of the national Anglophone committee, and the (regional) RDÉE s are the only organizations that are eligible for funding.

The document review and key informants indicate that funding applications go through the following process:

• Funding applications are examined by Service Canada program officers. The review is based on pre-established approval criteria, terms and conditions, and on general eligibility requirements (all based on the national committees' strategic plans).
• An internal review committee at Service Canada looks at the applications and recommends approval to the Minister. Applications that are not successful are returned to the eligible organization for reconsideration.
• Once approved, funding applications are inputted into the Common System for Grants and Contributions (CSGC) used by HRSDC and Service Canada. It should be noted that key informants from HRDSC and Service Canada reported that the CSGC does not enable users to monitor or compile results from the contribution agreements (the issues of results data and the lack of a computerized administrative database are examined in more detail in Section 3.1.5).

Adjustments made to the funding application approval process since the 2002-2003 formative evaluation of the predecessor program include the addition of:

• selection criteria and guidelines;
• analysis grids;
• more detailed report templates;
• clearer and more specific procedures; and
• an evaluation of the percentage of completed activities.

There has also been an improvement in the training and support provided by HRSDC and Service Canada to the signatories to the contribution agreements.

3.1.2 Program Objectives

Clarity and Understanding of Program Objectives

Evaluation Questions: Are the program's objectives formulated clearly and precisely? How effectively are the objectives communicated to and understood by the stakeholders?

The program objectives are expressed clearly and precisely in the Results-based Management and Accountability Framework (RMAF) for the Enabling Fund. A detailed narrative to the program logic model explains the short, medium and long-term objectives of the program. Key informants are also of the general view that the objectives of the Enabling Fund are expressed clearly and precisely.

Key informants stated there is good communication between Service Canada and the signatories to the contribution agreements. This process is supported by the monitoring visits made by Service Canada officers and their ongoing communication with the contribution agreements signatories. The signatories confirmed that they are satisfied with the levels of communication and that there is a good understanding of the Enabling Fund's objectives among stakeholders.

Measuring the Program's Objectives

Evaluation Question: To what degree are the program's objectives measurable?

Activity and partnership reports are produced monthly by the signatories to the contribution agreements. The document review shows that these cumulative reports:

• allow for the monitoring of progress made in implementing planned activities;
• provide clear information on the number of partnerships developed; and
• allow for the monitoring and measurement of the leveraging effects.

As part of this evaluation, the monthly reports were used to draw together and analyze statistics on the leveraging effects (as discussed in Section 3.2.4).

The key informants from HRSDC and Service Canada state that the lack of baseline data remains a significant shortcoming when it comes to managing the program. As well, they note that the program lacks a computerized administrative database that is able to compile results and provide an overall picture of the results achieved. They feel that the development of such a database would make it possible to properly analyze results and present relevant cumulative information (this issue is discussed further in Section 3.1.5).

Key informants, particularly federal officials, also note that it will always be challenging to measure performance with respect to broad social objectives (such as community vitality) because the imprecise nature of these types of objectives make them difficult to quantify and measure. In addition, the lack of baseline data on the issues affecting the communities complicates the measurement of results and the progress that is still needed.

3.1.3 Program Design and Development

Research Activities to Support the Program's Design and Development

Evaluation Question: In terms of program design and development, what policy research activities were carried out?

Under the RMAF for the Enabling Fund, research carried out by HRSDC and Service Canada, the contribution agreement signatories and other participating departments is one of the pillars of the results-oriented logic model. These research activities are to guide all the partners in the setting of priorities and to support the decision-making process. ^{Footnote 11}

Service Canada managers report that they analyzed the implementation of similar programs, to learn about best practices and lessons learned. As well, useful information sources include census studies from Statistics Canada, papers and publications from Canadian Heritage, as well as the work of the program's national committees.

Overall, however, the key informants from HRSDC and Service Canada report that policy research activities to support the design and development of the program are incomplete. According to Service Canada and HRSDC officers, a number of factors limit the usefulness of research activities to support the program's objectives. For example, the Enabling Fund is not recurrent and is time-limited, which does not encourage research into the subject. As well, program managers report that they are focused on identifying tangible results. They also note that the lack of a computerized administrative database increases the challenge of collating the results in a meaningful manner.

Regarding the development of policy directions, some key informants stated that this activity should originate with the national committees, given the mandates of those committees. ^{Footnote 12} They added that the national committees would increase their potential if the committees engaged in the development of policy directions in a more sustained fashion. The national committees do strategic planning, but some of the key informants feel that the national committees should adopt long-term strategic thinking and attempt to come up with a more focused vision to guide all their funded activities. This opinion is shared by the FCFA and the QCGN.

Methods Used to Consult with Communities on Program Design and Development

Evaluation Question: In terms of program design and development and service delivery, how were the stakeholders in the official language minority communities consulted?

The key informants indicate that the national committees are the preferred method for consulting with the communities, despite the lack of strategic content at the meetings. National committee meetings are held four times a year.

The key informants also indicate that numerous management tools have been developed with input from the communities (such as the monthly activity and partnerships reports). All key informants feel that stakeholders were fully involved at all stages of the consultation process. That involvement included validation of the program's management tools with members of the community. As well, the signatories to the contribution agreements report that they are satisfied with the consultation methods used by Service Canada.

3.1.4 Causal Link Between the Program's Objectives and its Activities

Relevance of Activities to the Objectives

Evaluation Question: Are the program's activities relevant, given the stated objectives?

The following evidence from the document review confirms the relevance of the program's activities with respect to the program's objectives.

• Apart from the challenges associated with measuring the long-term impact (i.e. on community vitality), the activities of the RDÉE s and CEDECs funded under the program appear to be appropriate and relevant to the program's objectives.
• Service Canada program officers analyze the funding applications and the proposed activities based on pre-established criteria designed to ensure that the program's objectives are reached (as discussed in Section 3.1.1).

Links Between the National Committees' Strategic Plans and the Program's Mandate

Evaluation Question: To what extent do the priorities established under the strategic plans of the national committees relate to the program's mandate and objectives?

The document review shows that there is a direct link between the priorities of the strategic plans of the national committees and the program's mandate and objectives. This was confirmed by the key informant interviews with program managers.

Links Between the Contribution Agreements and the Program's Mandate and Objectives

Evaluation Question: To what extent do the objectives of the contribution agreements relate to the program's mandate and objectives?

The document review shows that the grid used by Service Canada in analyzing funding applications is a compulsory step in the approval process. The grid demonstrates the link between the proposed activity and the program's mandate and objectives.

3.1.5 Data Collection

The 2002-2003 formative evaluation of the predecessor program identified a number of significant shortcomings in that program's data collection and processing system. To address these shortcomings, HRSDC identified a number of specific measures in its management's response to the 2002-2003 evaluation. In addition, in 2005 HRSDC brought in an RMAF that documented the commitments made by managers with respect to the Enabling Fund. The RMAF also included indicators and an accountability mechanism that complies with the mandate for the Enabling Fund.

Data Collection Tools

Evaluation Question: Does the Enabling Fund have a system or tools for collecting data?

Monthly reports are required from the signatories to the contribution agreements: an activity report, a partnership report, and another report on the community plans. In addition, annual reports are produced by RDÉE Canada and by the community table of the national Anglophone committee. These data collection tools were developed in consultation with the signatories to the contribution agreements. In addition, as discussed under the next evaluation question, the data collection process has been normalized (e.g. through report templates, coaching and training sessions).

The document review shows that the submitted reports are rich in descriptive elements (activities), but do not sufficiently emphasize results and performance measurement. The document review also found that the monthly capture of data from the reports is not being integrated by the program managers to provide an overall picture.

Although the Enabling Fund has data collection tools, the program managers reported experiencing difficulties in closely monitoring the program as a whole because of the lack of a computerized administrative database capable of collating/analyzing the program's results and providing an overall picture of the results achieved. Although a prototype of a computerized administrative database was created in the summer of 2007, the data processing problem identified in the case of the predecessor program in the 2002-2003 formative evaluation has carried over into the Enabling Fund.

Stakeholders' Data Collection Roles and Responsibilities

Evaluation Question: Are the roles and responsibilities of the stakeholders regarding data collection well defined?

Appendix D of the 2005-2006 and 2006-2007 contribution agreements clearly describes the roles and responsibilities of the signatories with respect to the collection of data by specifying the expected activities and deliverables. In addition, the strategy guiding the monitoring process for 2005-2006 deals with the production of monthly activity reports and a final report. For 2006-2007, the strategy focuses on the monthly production of activity reports, a partnership report, a report on the community plan, and a report on claims for the reimbursement of incurred expenses.

A number of activities were conducted to clarify the roles of each stakeholder. Report templates were created by the program managers working with the signatories to the contribution agreements in order to ensure that the tools were relevant. Training sessions were also offered to the signatories. Consequently, the data collection process has been formalized/normalized and a more rigorous accountability process launched. The key informant analysis indicates that all parties feel that progress has been made in this area.

In addition, a consistency grid has been developed and used by the Service Canada program officers to monitor activities and to identify when action needs to be taken in order to readjust activities.

Evaluation Question: Do the signatories to the contribution agreements collect data that are relevant to the program's activities?

The document review shows that the requested activity reports have been submitted on a regular basis especially since 2006. The document review also confirms the link between data collected and the program's activities (as illustrated by the analysis of activities in Section 3.2.2 and by the analysis of partnerships in Section 3.2.4).

As well, the document review reveals that the quality of the reports is improving. In winter 2007, the program managers arranged training meetings in order to improve the quality and the relevance of the information collected from the signatories to the contribution agreements (i.e. incomplete information, lack of reporting on results). The document review shows positive changes between the way that data were recorded in 2005-2006 and the recording of data since then, especially since the winter of 2007.

At the same time, however, the document review found that the activity and partnership reports remain descriptive, because the template does not emphasize performance measurement.

Evaluation Question: To what extent do the data collected allow for appropriate monitoring of the implementation of the activities and projects funded under the contribution agreements?

The document review shows that the activity and partnership reports have been designed to be cumulative, which makes it easier to observe the progression in the results. In addition, as noted above, the document review shows that the quality of the reports has improved over time, particularly since the winter of 2007. As well, the partnership reports are considered the preferred tool for use in identifying the partnerships that have been developed. For example, the structure of the reports allows for a thorough analysis of the partnerships, motives and reasons for success or failure.

However, the document review also shows that the accountability that has been built into these data collection tools is focused more on the implementation and delivery of the activities, and less on measuring performance through the use of performance indicators. The document review also reveals that some parts of these reports have not been completed, primarily in the community plans, and this tends to limit the thoroughness of the monitoring.

Evaluation Questions: What use do the program managers make of the collected data? Have they published the results obtained and progress made with respect to the program objectives, based on specific performance indicators?

Program managers indicate that the collected data are used to ensure proper management of the program and to monitor and control activities. Specifically, the program managers indicate that they use the collected data in preparing internal documents and in preparing documents required by Treasury Board Secretariat.

The program managers have not made public the results obtained and progress achieved with respect to the program's objectives based on the data collection instruments. As well, the monthly capture of data has not been the object of an overall integration by program managers. The program managers feel that the lack of a computerized administrative database makes it difficult to publish reliable and complete results and information on the progress achieved with respect to the program's objectives.

3.1.6 Performance Indicators

Evaluation Question: Are the performance indicators appropriate and measurable? Do the contribution agreements include performance indicators that relate to those of the program?

The document review shows that the Enabling Fund's RMAF includes precise objectives and performance indicators. As well, the objectives and performance indicators are appropriate and measurable.

Although the program managers maintain that the performance indicators are incorporated into the contribution agreements, the document review shows that the contribution agreements do not include any performance indicators. In other words the document review shows that the indicators contained in the RMAF are not supported by corresponding indicators in the contribution agreements. The indicators contained in the contribution agreements relate more to the activities undertaken than to the results obtained. The document review did show, however, that the monthly reports provide information on the monitoring of activities, partnerships and community plans.

Evaluation Question: To what extent do the data collected support performance measurement, either at the program level or the contribution agreement level? Do the signatories to the contribution agreements monitor performance measurement?

The document review shows that the monthly reports provide cumulative data on the monitoring of activities and partnerships. This makes it possible to quantify the program's leveraging effect towards partnerships and the diversification of funding sources (as illustrated in Section 3.2.4).

However, as reported above, the document review shows that the contribution agreements do not include any performance indicators, and the monthly reports do not provide information on the results achieved by the activities undertaken or by the leveraging effects. Therefore, the data collected do not go far enough to provide information on the results achieved.

The signatories to the contribution agreements produce the reports required under the program, but (as discussed above) the reports do not emphasize performance measurement. However, some progression toward a results-based accountability can be seen. For example, in Manitoba, the Conseil de développement économique des municipalités bilingues du Manitoba claims that it is able to measure the leveraging effect for each dollar invested. As well, some RDÉE s stated that they carry out a more complete form of performance measurement than what is required, using their own indicators. It was also pointed out that work has been done by RDÉE Canada on a new logic model that would be more suited to the program's accountability requirements.

3.1.7 Roles and Capacities of the Signatories to the Contribution Agreements

Evaluation Question: Have the signatories to the contribution agreements produced any research reports?

Key informant interviews with the CEDECs and RDÉE s reveal that the signatories to the contribution agreements have produced a variety of research reports including community surveys, demographic and community profiles, employability and human-resources studies, statistical reports, economic diagnostics, studies on youth out-migration, studies on entrepreneurship and services available in French, research on leveraging effects, feasibility and market studies, research on essential human resources skills, and cost-benefit analyses.

Evaluation Question: Have these reports contributed to an improvement in the knowledge of the socio-economic issues faced by official language minority communities?

Most of the signatories to the contribution agreements state that their reports and research work present a true picture of the situation, help to better understand the needs of the communities, and help to better target future actions.

Federal officials observed that the socio-economic profiles produced by the signatories have helped improve the knowledge of economic issues, especially issues associated with community decline, the dynamics of education and the out-migration of youth. Therefore these key informants feel that the research has served to target activities more directly to identified needs.

Evaluation Question: How do the program managers and the signatories to the contribution agreements use the research results?

Signatories to the contribution agreements indicate that the CEDECs and RDÉE s use the research results to identify community needs, plan and promote their activities and projects, establish partnerships, and follow the progress of their situations. As well, the signatories feel that the research products help justify the activities and projects identified through the funding application process to various funding partners. In particular, they feel that the research products help persuade other partners to become involved. Federal officials also feel that the research results give the added credibility needed to support the proposed actions.

Evaluation Questions: Is there a direct relationship between the socio-economic profiles that have been developed and the activities carried out within the framework of a contribution agreement? Based on the activities and projects that have been funded, to what extent do the signatories to the contribution agreements play a role in achieving the program's objectives?

The document review shows that the activities proposed under the contribution agreements are developed based on information on socio-economic conditions. The document review also shows that the activities carried out by the RDÉE s and the CEDECs are focused directly on community capacity building, community economic development, and human-resource development.

In their activity reports, the signatories to the contribution agreements identify the causal link between activities, program objectives and the needs of their communities. This overall finding is confirmed by the Service Canada program officers who ensure (when reviewing applications for funding) that the activities and projects proposed by the signatories directly support the program's objectives.

Evaluation Questions: Are the signatories to the contribution agreements equipped and trained to carry out their roles? What are the guiding principles, the operational guidelines and management tools available to the signatories to the contribution agreements? Does Service Canada offer functional direction and appropriate support to the signatories of contribution agreements? Are the signatories satisfied with the support provided by Service Canada? What about the operational guidelines and management tools made available to them by Service Canada?

The signatories to the contribution agreements state that they are well equipped and trained to carry out their roles, but a number of them noted a lack of human resources and a lack of funding. The program managers confirmed that the level of capacity and maturity of the eligible organizations varies from one organization to another.

Appendix A of the contribution agreements spells out the operational directives and what is expected of each of the signatories. Appendix C of the agreements describes the conditions for receiving funding instalments, and Appendix D describes the roles and responsibilities of the parties.

In addition, program managers worked with the signatories to the contribution agreements to develop various management tools to support the program-related work of the signatories. These tools include a guide to the completion of funding applications, and standard templates for the activity and partnership reports. The signatories to the contribution agreements have also been given training in the use of these tools.

As well, the program managers and officers arrange follow-up visits. They also provide ongoing coaching (direction and support) for the signatories to the contribution agreements, and arrange training sessions.

At the network level, the meetings of RDÉE directors and the sectoral tables (youth, communication, rural development, tourism, and the knowledge-based economy) are additional management tools that help to clarify and remind those involved of the principles of management and the operational directives.

The signatories to the contribution agreements reported a general level of satisfaction with the support provided by Service Canada. The signatories also indicate that they appreciate that the management tools were developed with their cooperation. Most of the signatories are satisfied with the availability of their Service Canada officers, but some of them would like to see greater consistency in the advice given out by Service Canada, a relaxation of the administrative processes and an increased presence of Service Canada officers in the field.

Evaluation Question: What is the level of organizational stability of the organizations that are eligible for funding under the program? Are there any issues related to organizational capacity and staff retention? Do these issues affect implementation of the program?

Regarding organizational capacity, the HRSDC and Service Canada managers indicated that the level of capacity and maturity of the organizations eligible for funding under the Enabling Fund varies from one organization to the next.

Regarding staff retention, program managers indicate that retention and succession are problem areas, especially in the more remote regions. There are reports of a high staff turnover rate in some RDÉE s in the more remote regions. Within the RDÉE network, it has been observed that many employees (after they have been trained) leave to go to the public service, where employment conditions are better and there is greater job security. The issue of salary competitiveness is seen as important because the RDÉE s have a ceiling on the maximum wage increase of 3% per year. The CEDECs do not have a particular problem with staff retention; however, some CEDEC staff reported fatigue and exhaustion of volunteers.

Evaluation Questions: Is the organizational structure of the eligible organizations representative and inclusive of all the players in the areas of human resource development and community economic development? Are eligible organizations duplicating the work of other existing community organizations?

The organizations eligible for funding under the Enabling Fund generally claim that their organizational structures are representative and inclusive of the various sectors of their communities. These organizations rely on boards of directors that are representative of their communities, on representative advisory committees or councils, on community committees, and on community consultation tables and forums.

This general view is not shared by the QCGN and the FCFA . These two groups maintain that the inclusive and representative nature of the program's organizations needs to be improved.

In general, the organizations eligible for funding under the Enabling Fund feel they are not duplicating the work of other community organizations. Specifically they feel that they have an exclusive mandate that sets them apart from other organizations.

3.2 Achievement of Short- and Medium-term Objectives

This section uses the evidence gathered by the evaluation to examine the progress made towards achieving the short- and medium-term objectives of the Enabling Fund.

3.2.2 Immediate Outcome 2 and Intermediate Outcome 1: Increased Capacity of Contribution Agreement Signatories, RDÉE s and CEDECs

This section uses the evidence gathered by the evaluation to examine the progress made towards achieving the immediate outcome of increased capacity of contribution agreement signatories to act as agents of human resource and community development.

This section also examines the progress made towards achieving the intermediate outcome of increased capacity of the RDÉE s and CEDECs to develop or assist in the development of community plans and projects.

Evaluation Question: What is the inventory of funded activities and their expected impacts at the local level?

Completed activities are inventoried cumulatively on a monthly basis in the activity and partnership reports submitted by the signatories to the contribution agreements. The document review indicates that activities conducted by the signatories include forums, community consultation sessions, community projects and plans, and support for community organizations. Specific examples of activities drawn from the monthly reports include:

• a study of potential resources and development possibilities in six municipalities in the Grand Caraquet region (N.B.);
• creation of a network of Francophone and francophile business people in the Miramichi region (N.B.) (consisting of forty members and growing);
• a strategic plan to integrate more Francophones into the political and economic structure of Timmins (Ont.);
• a feasibility study and business plan for a new French-language bookstore in Sudbury (Ont.);
• a labour-force adjustment study on the needs of the agricultural sector (Ont.);
• training in cooperatives and different types of community development (B.C.); and
• an economic partnership agreement with the province of Manitoba (for 3 years) involving three main components: (1) the Community Development Corporation (i.e. creation of one rural economic development officer position); (2) youth entrepreneurship (i.e. entrepreneurship and mentoring camp for young entrepreneurs); and (3) an international project (i.e. development of business immigration and exports).

The monthly reports submitted by the signatories to the contribution agreements do not provide information on the impacts expected or achieved in the field. Therefore, it is not possible to summarize the potential impacts of activities based on the data collection tools developed by HRSDC and Service Canada and the signatories to the contribution agreements.

Evaluation Question: What is the geographic scope of activities, projects and partnerships on a provincial/territorial scale?

It is not possible to compile information on the geographic scope of their activities and projects from the activity and partnership reports submitted monthly by the signatories to the contribution agreements. However, the activities of the CEDECs and RDÉE s appear to extend to all official language communities.

• The eleven CEDECs have a presence in all regions of Quebec where there are concentrations of Anglophone populations, and they respond to requests at the regional level. Together they cover the entire province of Quebec.
• The RDÉE s are active in the other provinces and territories of Canada. They concentrate their efforts in Francophone regions and elsewhere where required.

Evaluation Question: Do RDÉE s and CEDECs have the planning tools and expertise needed to develop community plans and projects? Are they approached by community members or organizations? Have any RDÉE s or CEDECs developed community plans or projects? If so, of what type, nature and scale?

Key informants in the field indicate that the RDÉE s and CEDECs have planning tools and expertise such as professional development workshops, toolkits and resource inventories.

• CEDEC key informants state that they possess the tools and skills to do the job. They feel they are skilled in organizing forums and workshops, and in drawing up entrepreneur profiles. As well, they have access to tools such as volunteer pools, community profiles, geographic integration modules and networking technology guides.
• RDÉE s have socio-economic profiles, action plans, logic models and skills profiles of development practitioners. They also arrange theme-based training sessions, forums and community committees.

The key informants in the field indicate that the RDÉE s and CEDECs have been approached both formally and informally by corporations, community organizations and non-profit organizations to prepare strategic and business plans.

According to most RDÉE key informants, the RDÉE s participate in developing community plans and projects of varying scope and nature in the four activity sectors: youth, knowledge-based economy, rural development, and tourism. These projects and plans cover, for example, projects aimed at youth retention, labour force reintegration, literacy, economic diversification and employability.

Evaluation Question: Do the contribution agreement signatories have strategic and operational plans?

Both the key informant analysis and the document review show that the signatories to the contribution agreements have strategic and operational plans. Furthermore, both networks have strategic plans focused on performance and results-based management. These plans mainly include a discussion of objectives, activities, performance indicators, deliverables, timelines and partners. For example:

• 2006-2009 strategic plans and a five-year plan were prepared by the Yukon RDÉE ; and
• the Newfoundland and Labrador RDÉE has a five-year strategic plan, and the operational plan is updated regularly.

Evaluation Question: Does RDÉE and CEDEC leadership support the commitment of members and the mobilization of community resources?

Key informants in the field provided the following examples to indicate RDÉE and CEDEC leadership in supporting the mobilization of community resources:

• training workshops for sector committees;
• the development of numerous community projects; and
• assistance with social enterprise building.

Some community partners stated that RDÉE and CEDEC involvement played a key role. Several partners claimed that without the proactive involvement of the RDÉE s and CEDECs, many projects would never have seen the light of day.

Evaluation Question: Do contribution agreement signatories share best practices? How is this done?

Ongoing discussions and activities to share best practices take place within the RDÉE s and within the CEDECs.

• Among CEDECs, best practices are communicated through newsletters, meetings and contacts between CEDECs. One section of the community table's annual report is reserved for best practices.
• In the case of the RDÉE s, best practices are shared at the national issue tables and sector tables.

Discussions between the RDÉE s and CEDECs to share best practices had only just begun at the time of the evaluation.

Evaluation Question: Have eligible organizations helped communities create and improve their own social and economic structures in support of building community capacity?

In numerous initiatives, eligible organizations have helped communities to create and improve their own social and economic structures in support of building community capacity.

• One example is the closure of a textile factory in Huntingdon. Working with other stakeholders, the South-West CEDEC organized a public consultation in Huntingdon to find new directions for the economy. The effect was to trigger a large collective movement. Workers developed a plan to find new directions for the local economy. Six-hundred of the 650 laid-off workers found other jobs or returned to school.
• Examples of some of the projects supported by RDÉE s also illustrate the broad range of these initiatives. These examples include a fair-tourism travel agency, business expansions, the establishment of an early childhood education centre in the Yukon, and l'Écho d'un peuple (a large-scale annual cultural event in Ontario).

Despite these and other examples of eligible organizations helping communities to establish and improve their own social and economic structures, the information needed to quantify the benefits is not available.

3.2.3 Immediate Outcome 3: Increased Knowledge of Issues Affecting Official Language Minority Communities

This section examines the progress made towards achieving the immediate objective of increased knowledge within the national committees, RDÉE s, CEDECs, and federal departments and agencies of the issues affecting official language minority communities.

Evaluation Question: Do the contribution agreements show a detailed knowledge of local human resource and community economic development issues?

Appendix A of the contribution agreement clearly sets out the problem areas and the interpretations which underlie the targeted activities. The activities described in the contribution agreement support community capacity building, community economic development, and human resource development. As well, contribution agreement signatories must demonstrate a thorough knowledge of the areas where they hope to intervene.

Evaluation Question: What types of community consultations are carried out when preparing funding applications?

Signatories to the contribution agreements must provide community profiles and must provide a community plan in their accountability reports.

Federal officers confirm that community consultation is achieved through forums, surveys, meetings and field assessments of project relevance. This range of methods allows for ongoing community contributions and feedback.

Evaluation Question: On the Francophone side, are the community profiles that have been developed under the program used as key planning tools by contribution agreement signatories?

The community profiles are used by RDÉE s to help define the socio-economic issues and to justify proposed activities. Key informants also indicate that the profiles could be used to help measure changes in the socio-economic context.

Evaluation Question: On the Anglophone side, what labour force information tools are used for planning? Based on current outlooks, does the socio-economic diagnostic developed in 1999-2000 still reflect the needs of Anglophone communities?

The CEDECs use information from various sources. They use Service Canada's economic profiles, statistics from Emploi-Québec and local employment centres, sector and employment perspectives posted on federal Web sites, and studies they have conducted in specific sectors.

The socio-economic diagnostic developed in 1999-2000 is considered to be obsolete by the key informants.

3.2.4 Immediate Outcome 4: Improved Access to Existing Federal Programs

This section examines the progress made towards achieving the immediate outcome of improved access to existing federal programs for official language minority communities.

Evaluation Question: What steps have been taken by HRSDC and Service Canada to improve the knowledge that community organizations have of existing federal programs? What concrete actions have been taken by HRSDC and Service Canada to help contribution agreement signatories mobilize funding resources in other federal departments and agencies and in the private sector? Are these measures limited to organizations eligible for funding under the Enabling Fund?

Federal officials from HRSDC and Service Canada identified no systematic activities to support this objective. For example, Service Canada officers stated that they publicize the various funding possibilities, but they recognize that there is no formal process to support this objective. Federal officials feel it would be useful to have such activities defined and implemented.

In the absence of a well-defined horizontal process, the representatives of federal agencies and departments sitting on the national committees act as informants and accept responsibility for communicating information on their programs on a regular basis. The key informants also stated that meetings of a more official nature with federal representatives help communicate this knowledge and also result in specific aid agreements being handled more rapidly. In addition, Canadian Heritage has drawn up an inventory of programs available to official language minority communities. This information is available on the Canadian Heritage Web site and is a source of information for the signatories to the contribution agreements.

This evaluation was not able to determine whether these measures are limited to those organizations that are eligible for funding under the Enabling Fund.

Evaluation Question: Do eligible organizations have a better knowledge of and improved access to federal programs?

The analysis of key informants interviews revealed that five CEDECs (out of eleven) and eight RDÉE s (out of twelve) feel they are now more aware of federal programs. Most RDÉE s and CEDECs state that they have better access to federal programs, but they feel this is due to their own efforts and not due to the established information-sharing mechanisms.

Evaluation Question: Do eligible organizations sign contribution agreements with federal departments and agencies other than HRSDC? Based on contribution agreements signed under the Enabling Fund, what is the scale of contributions from other federal departments and agencies? Which programs were used? Was any progress noted? Apart from the Enabling Fund, what is the scale of investments from the public and private sectors that contribution fund signatories have succeeded in attracting?

The document review indicates that, in 2006-2007, RDÉE s and CEDECs signed other contribution agreements with various federal agencies and departments: 98 were for funding, and 91 involved in-kind contributions.

Using the partnership reports, the document review shows that CEDECs and RDÉE s obtained the equivalent of $19.5 million in financial funding and $5.3 million in-kind contributions, for a total of $24.8 million as a result of partnerships developed under the Enabling Fund (see Table 3).

A further examination of the records submitted by the partnership reports shows that contributions (financial plus in-kind) from federal partners amount to $7.2 million out of the total of $24.8 million. Contributions (financial plus in-kind) from non-federal sources (public, private, provincial) account for $17.5 million of the total of $24.8 million (see Table 4).

Although this analysis shows that the signatories to the contribution agreements are accessing other federal programs, it is difficult to determine what portion of these contributions can be attributed to the Enabling Fund because (as noted above) the organizations felt that they had better access to federal programs due to their own efforts, and not due to the established information-sharing mechanisms.

At the time of the evaluation it was too early to draw any meaningful conclusions regarding progress in the scale of contributions from other federal agencies and departments, because data presented in Table 3 and Table 4 were taken from partnership reports (which have only been produced since April 2006). As further monthly reports are received, however, the activity and partnership reports will make it possible to track this progress over time.

• Source from the above table Source: Statistics derived from CEDEC and RDÉE partnership reports.

• Source from the above table Source: Statistics derived from CEDEC and RDÉE partnership reports.

Evaluation Question: What impact has the introduction of the government's Action Plan for Official Languages had on organizations eligible for funding with respect to improved access to other federal programs?

Key informants were divided on whether the introduction of the government's Action Plan for Official Languages had an impact on improving access to federal programs by organizations eligible for funding under the Enabling Fund.

• Many key informants note that the Action Plan gave added impetus to official language issues. They feel that this renewed interest attracted attention from federal departments and led to the involvement of the Privy Council Office.
• Other respondents describe the Action Plan as weak on economic development and see no impact on the operations of the Enabling Fund.

3.2.5 Intermediate Outcome 2: Increased Mobilization of Resources and Investments

This section examines the progress made towards achieving the intermediate outcome of increased mobilization of resources and investments from public sectors and consolidated funding partnerships.

Evaluation Question: What is the nature and scope of existing funding partnerships created as a result of funding from the Enabling Fund?

The nature of the partnerships created by the RDÉE s and CEDECs is not fully described in the partnership reports. Nevertheless, the available documentation shows that CEDECs and RDÉE s obtained the equivalent of $19.5 million in financial contributions and $5.3 million in-kind contributions, for a total of $24.8 million (as shown in Table 3). The available documentation also shows that contributions from non-federal sources (public, private, provincial and other) accounted for $17.5 million of the total of $24.8 million (as shown in Table 4).

Evaluation Question: What difficulties do contribution agreement signatories encounter in developing partnerships?

Slow administrative processes are seen as a problem by contribution agreement signatories. The timing for submitting funding applications varies from one department to another, according to the budget cycles, and this entails considerable advance planning. Given that financial stability remains a concern of the contribution agreement signatories, they maintain that it is important to standardize funding approaches and procedures among federal departments.

It was also noted that in some regions it is not possible to communicate in French with certain federal institutions.

Evaluation Questions: What concrete actions have been taken by contribution agreement signatories to diversify funding sources? What role do the national committees play in this?

The RDÉE s and CEDECs organize numerous activities aimed at diversifying funding sources including tri-partite tables, communication strategies, collaborative committees and various funding appeals.

• The CEDECs have contacted various partners in an attempt to diversify their sources of funding. As they have no charter, individually they can legally only obtain in-kind contributions. Some federal, provincial and local partners have occasionally contributed in this way. Examples include Economic Development Canada, Industry Canada, the Conférence régionale des élus, Place aux Jeunes, and Carrefour Jeunesse Emploi.
• The RDÉE s have approached federal, provincial and municipal governments as well as regional economic development agencies. Furthermore, some RDÉE s have succeeded with private-sector approaches. For example, RDÉE Alberta claims that 15% of its funding comes from the private sector.
• Following a feasibility study, the Francophone community table plans to establish a Fondation canadienne pour le développement économique et de l'employabilité, under RDÉE Canada, to access private-sector funding.

As noted in Section 3.2.4, in the absence of a well-defined horizontal process to help improve signatories' access to existing federal programs, federal agencies and departments sitting on the national committees communicate information on their programs on a regular basis. However the RDÉE s and CEDECs feel that the national committees do little to help them diversify their funding.

3.3 Horizontality

This section examines issues with respect to engagement and partnership across government agencies and departments and across community groups.

Evaluation Questions: In terms of program design, what are the horizontal aspects of the Enabling Fund?

The leveraging effects of partnerships and investments (as discussed in Section 3.2.4) represent tangible results of the involvement of partners from the public, private, municipal and community sectors.

The Enabling Fund was designed to ensure horizontality through two main features:

• the governance structure of the Enabling Fund includes the two national committees. These two committees bring together community representatives and representatives of various federal departments and agencies; and
• the inclusive structure of the RDÉE s and CEDECs ensures community representation.

Regarding the effectiveness of these program design features, however, the key informants provide the following points.

• To strengthen horizontality, the operations of the national committees must be improved through greater emphasis on concrete actions and follow-up. As well, it is important to have a clear orientation and to target precise results and objectives.
• The FCFA and the QCGN have reservations as to how well the program's horizontality works in the field. For example, the FCFA notes that the RDÉE s are not the only structures representing communities, and that a more inclusive and participatory approach involving other community representatives should be considered. The QCGN shares this opinion and points out that in the outlying regions of Quebec there is a certain amount of duplication among actions by CEDECs, employment centres and community futures development corporations, as they are all working in the same sector.

Evaluation Questions: What concrete actions have been taken by the federal table of the national committees, HRSDC and Service Canada to ensure that there is internal and external coordination regarding the harmonization of federal interventions in official-language communities?

The protocol signed between federal departments and agencies and the community table of the national committees sets out the purpose of the agreement, the principles for collaboration and mechanisms for implementation. HRSDC provides administrative support to the national committees. At the same time, however, a number of key informants mentioned the importance of improving how the national committees operate.

Evaluation Question: How does achieving Enabling Fund objectives contribute to achieving the objectives of the Action Plan for Official Languages?

Approximately half ^{Footnote 14} of the key informants who are federal officials believe that the Enabling Fund directly supports the Action Plan's community development objectives.

The Action Plan mid-term report, entitled Update on the Implementation of the Action Plan for Official Languages (Privy Council Office, 2005), encourages federal institutions to pool their expertise in order to assist the communities. The RDÉE s and CEDECs are also encouraged to apply to programs other than the Enabling Fund for complementary activities having exclusively economic development mandates.

Evaluation Questions: To what extent has awareness of community needs, acquired through the Enabling Fund, led to increased awareness of the same needs among other federal departments and agencies?

The key informants who are federal officials indicate that all of the Enabling Fund's activities and the management products (such as the various reports and economic profiles) have led to increased awareness and recognition of community needs among federal agencies and departments. In particular, the horizontality encouraged by the Enabling Fund has helped communities get their message across. Much more is heard on the subject of community economic development than was previously the case in these communities. Regional issues are better understood. For example, Industry Canada now has detailed knowledge of 902 rural and urban communities. These positive results are attributable to all of the initiatives described.

4. Conclusions

This section provides a summary of the main findings and the overall conclusions arising out of the formative evaluation of the Enabling Fund.

4.2 Achievement of Short- and Medium-term Objectives

4.3 Horizontality

The Enabling Fund is designed to ensure horizontality by funding the coordination of the two national committees which bring together community representatives and representatives from various federal agencies and departments. In addition, the inclusive structure of the RDÉE s and CEDECs ensures community representation.

The leveraging effects in terms of partnerships and investments represent tangible results with the involvement of partners from the private, public, municipal and community sectors. As well, federal officials indicated that the horizontality encouraged by the Enabling Fund has helped communities to get their message across.

At the same time, however, key informants provided a number of suggestions for improving the effectiveness of the program design features related to horizontality. Their suggestions included:

• increase the strategic content of the national committees;
• improve the operations of the national committees for example by placing greater emphasis on concrete results and follow-up; and
• consider adopting a more inclusive and participatory approach involving other community representatives.

5. Recommendations

Based on the results of the formative evaluation, the following recommendations for follow-up action are set out.

Greater effort and resources should be devoted to:

• Building and maintaining the internal capacity of Contributions Agreements Signatories and the CEDECs. Particular emphasis is required on:
• addressing issues relating to staff retention, succession planning and decreasing reliance on support received from volunteers (in the case of the CEDECs particularly); and
  • securing additional funding from diverse sources (Federal, provincial and private sector) to support organizational activities.
  • Improving the data collection and accountability processes. It is recommended that standardized data collection templates be developed and used by Contribution Agreement Signatories to support the accurate capture of program results and monitoring information into an electronic database at HRSDC. Such an electronic database will serve as a pillar for reporting, monitoring and evaluation activities.
• Improving the operations of the national committees by:
  • increasing the strategic content of discussions at the national committee meetings and by placing greater emphasis on achieving concrete results, including follow-up; and
  • adopting a more inclusive and participatory approach involving other community partners that reflect particularly regional diversity.