Evaluation of the Building Communities through Arts and Heritage for the period 2011-12 to 2014-15

October 17, 2016

Cette publication est également disponible en français.
© Her Majesty the Queen in Right of Canada, 2016
Catalogue No. CH7-10/2016E-PDF
ISBN 978-0-660-23189-1

Purpose

This evaluation report is to be used for performance monitoring and measurement purposes by the Department, and to inform policy and program improvements and any future update to the program’s Terms and Conditions (Ts&Cs).

Background and context

The BCAH program was launched as a new program in the Department of Canadian Heritage in 2007-2008 in response to the 2007 federal budget which expressed the intent to support local arts and heritage festivals and small capital projects that placed an emphasis on local engagement. At the time, the program was comprised of two components: Local Arts and Heritage Festivals and Community Historical Anniversaries Programming. In August 2009, the Community Historical Anniversaries Legacy Fund component was added to support larger capital projects that involve the restoration, renovation or transformation of existing buildings and/or exterior spaces. All three components place an emphasis on engaging Canadians in their communities through the performing and visual arts and in the expression, celebration and preservation of local heritage.

In 2014-2015, the program was a subprogram of Program – Engagement and Community Participation presented in the PCH Program Alignment Architecture (PAA) and contributed to Strategic Outcome 2: “Canadians share, express and appreciate their Canadian identity”.

Objectives and outcomes

The overall objective of the BCAH is to engage citizens in their communities through the performing and visual arts and in the expression, celebration and preservation of local historical heritage.^{Footnote 1} The program’s expected results, are articulated in the program’s Terms and Conditions (Ts&Cs) (2013).

Immediate outcomes

• Local organizations receive financial resources to carry-out local festival, community anniversary and / or legacy projects in their communities.

Intermediate outcomes:

• Local partners within the community provide support to funded local festival, community anniversary and / or legacy projects.
• Local citizens are provided with opportunities to engage in their communities through local arts and heritage.
• Local artists, artisans and/or heritage performers are provided with opportunities to engage in their communities through local arts and heritage.
• Local citizens have opportunities to be exposed to local arts and heritage.

Ultimate outcomes

• Citizens across the country are engaged in their communities through local arts and heritage.

In the long term, these outcomes are expected to support the PCH strategic outcome: Canadians share, express and appreciate their Canadian identity – which, aims to foster a stronger Canadian identity through active, engaged, inclusive citizenship and recognition of the importance of both linguistic duality and a shared civic identity. Results support the Departmental mandate in which Canadians take full advantage of dynamic cultural experiences, celebrate their history and heritage and participate in building creative communities.

Component I – Local Arts and Heritage Festivals (Local Festivals)

The Local Arts and Heritage Festivals component provides funding in support of festivals, events and activities that engage Canadians in their communities through recurring public presentations of local artists and/or of local historical heritage. This component provides funding of up to 100 percent of eligible expenses to a maximum of $200,000 for recurring festivals that present the work of local artists, artisans or performers of local historical heritage; actively involve members of the local community and are intended for and accessible to the general public. Over the four years covered by the evaluation, Gs&Cs expenditures averaged $10.6 million per year.

Component II – Community Historical Anniversaries Programming (Community Anniversaries)

The Community Historical Anniversaries Programming component provides funding in support of non-recurring events and activities that engage Canadians in their communities through the commemoration of major anniversaries of significant local events and/or persons. This component provides funding of up to 100 percent of eligible expenses to a maximum of $200,000 for non-recurring events, related activities and minor capital projects that commemorate a significant local historical event or pay tribute to a significant local historical personality; mark a 100th anniversary or greater in increments of 25 years (e.g. 125th, 150th); present the work of local artists, artisans or performers of historical heritage; actively involve members of the local community; and are intended for and accessible to the general public. Eligible capital projects such as community art projects, restoration of objects, community history books, statues and murals can be funded up to a maximum $25,000. Over the four years covered by the evaluation, Gs&Cs expenditures averaged $2.7 million per year.

Component III – Community Historical Anniversaries Legacy Fund (Legacy Fund)

The Community Historical Anniversaries Legacy Fund component provides funding in support of capital projects that engage Canadians in their communities through the commemoration of major anniversaries of significant local events and/or persons. The component provides funding of up to 50 percent of eligible expenses to a maximum of $500,000 for community capital projects that commemorate a significant local historical event or pay tribute to a significant historical personality; mark a 100th anniversary or greater, in increments of 25 years (e.g. 125th, 150th); involve the restoration, renovation or transformation of existing building and/or exterior spaces with local community significance; encourage arts and heritage activities in the community and are intended for and accessible to the general public. Over the four years covered by the evaluation, Gs&Cs expenditures averaged $3.8 million per year.

In 2013, to align with Government of Canada (GC) priorities, the program’s Ts&Cs were amended for the Community Anniversaries and Legacy Fund components to include the commemoration of the 75th anniversary of locally significant events directly related to the Canadian participation in World War II.

Program management, governance, target groups, key stakeholders and delivery partners

BCAH is overseen by the Citizen Participation Branch in the Citizenship, Heritage and Regions Sector at PCH National Headquarters in Gatineau. PCH’s five Regional Offices (Atlantic, Quebec, Ontario, Prairies and Northern and Western) have primary responsibility for the delivery of the Local Festivals and Community Anniversaries components. The Legacy Fund is delivered through the Community Engagement Directorate ( CED ) at National Headquarters.

In addition to delivering the Legacy Fund component, CED plays a significant role in supporting the delivery of the Local Festivals and Community Anniversaries components including: the development and distribution of learning, information and support materials, program guidelines and application forms; supporting the regional and national review committees; reading/monitoring all files; and designing (including testing phases and quality control) and administering the funding formula for Local Festivals and Anniversaries. Additionally, National Headquarters participates in and implements corporate initiatives such as the enterprise online system (EOS), manages the overall budget, develops and updates performance measurement tools and procedures, and collects, analyzes and reports on results for all components.

Program resources

For the period covered by the evaluation, the program had total Gs&Cs expenditures of approximately $68.3 million and O&M expenditures of approximately $15.8 million (Table 1 ).

Table 1: Budgeted and actual expenditures 2011-2012 to 2014-2015

Source: Finance data

Evaluation context

2011 evaluation of BCAH

The program was evaluated in 2011. This evaluation covered the period from 2007-2008 through 2010-2011. It included limited evaluation coverage of the Community Historical Anniversaries Legacy Fund component as it was a relatively recent addition to the program (launched in August 2009) at the time of the previous evaluation.

The evaluation of the BCAH validated the need for continued federal support for local festivals and community anniversary events and projects. The evaluation found that program components were delivering results, but also pointed to opportunities for improvement to program design and delivery. Recommendations were made to explore ways to improve the timeliness of the funding response to recipients and to improve program efficiency by streamlining and automating the application processes. In addition, the evaluation found the potential for overlap between BCAH and several other PCH programs, such as the Canada Arts Presentation Fund ( CAPF ), Cultural Capitals of Canada, Canada Cultural Spaces Fund ( CCSF ) and the Celebration and Commemoration Program (CCP) as they funded similar events and activities; and therefore recommended examining opportunities for efficiency gains between these programs.

As follow-up to the recommendations the program took a number of actions to improve program efficiency including simplifying applications and guidelines, and participating in the Gs&Cs modernization initiative. Reviews of the potential overlap between BCAH and other PCH programs, completed in 2014, concluded that although PCH programs may share clients or provide funding for similar expenses, they have different objectives and are designed to fill a specific niche in the need for support. It was concluded that BCAH was complementary to other PCH programs.

There have been no substantial changes to the program since the 2011 evaluation.

Evaluation scope, timing and quality control

The evaluation of BCAH was conducted by ESD with components contributed by the PRG. It covered the period from 2011-2012 through 2014-2015 and addressed the five core evaluation issues of relevance and performance, in accordance with the TBS Directive on the Evaluation Function (2009) (See Appendix A). The evaluation also looked at the program’s design and delivery, areas for improvement, and performance measurement.

Evaluation questions

An evaluation framework, organized by evaluation issue, with a listing of the methodologies to be used to address each issue, was developed to support the evaluation (Appendix B).

Evaluation methods

Multiple lines of evidence were used to increase the reliability of the findings. Lines of evidence were comprised of primary data sources including key informant interviews, on-line surveys of program recipients and unsuccessful Local Festivals applicants and case studies. Secondary data sources included reviews of the literature, documents and the program’s administrative databases. The observations, findings and conclusions of the report are based on more than one line of evidence, unless otherwise stated. The evaluation included the following data sources:

• Literature review: The PRG provided a review of pertinent literature on behalf of ESD and a scan of similar programs in other jurisdictions. The evaluation team augmented PRG’s review and incorporated the results into their analysis.
• Review of program documents and administrative databases: Documents reviewed included key governmental documents (e.g., Throne Speeches and federal Budget extracts), departmental documents (e.g., Department Performance Reports ( DPR s), Reports on Plans and Priorities (RPPs)), and program-related documents such as Ts&Cs, application guidelines and forms, correspondence, and minutes of meetings between national headquarters and regions. This was combined with a review of two program databases: an extract from the Grants and Contribution Information Management System (GCIMS) which yielded financial, project and recipient-specific information; and a performance database which contained outcome data collected by the program from recipients’ final reports.
• Key informant interviews. Interviews were conducted by ESD with 44 key informants and included program employees from national and regional offices (15), funded organizations (12), unfunded applicants (6), municipal representatives (3), representatives from provincial funding programs (5) and experts (3). The evaluation team conducted the interviews and the analysis and reporting of findings.^{Footnote 2}
• Case studies: Seven case studies were conducted, involving three legacy projects – two involved recognized heritage buildings and one a municipal venue enhancement. Two of the legacy projects had also received funding through the Community Anniversaries component. Four case studies were of Local Festivals projects (1 small, 1 medium and 2 large). The methodologies used were interviews and document, website and file reviews. Interviews included executive directors/lead volunteers of recipient organizations (6), PCH staff (7), volunteers or artisans (3) and municipal representatives (3).
• Surveys of recipients and unfunded festival applicants: ESD and PRG undertook online surveys of organizations funded under each component. A fourth survey was undertaken of festival applicants who applied for but were unsuccessful in receiving funding.^{Footnote 3} The evaluation team conducted the analysis of the data compiled by PRG.^{Footnote 4}

Methodological limitations

Limitations were mitigated by the use of a multi-method approach to generate evidence on the evaluation questions from more than one line of enquiry and from different (internal and external) perspectives. These limitations included the following:

• The complex scope of the program. BCAH consists of three components, with different activities and scope but with the same objectives and outcomes. All three program components have been considered throughout the evaluation and, where possible and appropriate, findings are discussed in relation to a particular component.
• Partial outcome data for Legacy Fund projects approved in 2014-2015. Projects can span several years before a final report with project outcome data is available. Data collected through surveys and interviews provided additional data for 2014-2015 legacy projects.
• Limited information to assess the ultimate outcome of BCAH funding. The program does not collect data on the ultimate outcome of the program. Anecdotal evidence from the surveys and interviews provided some information on longer-term outcomes.
• Administrative data on funding sources for BCAH -funded projects, other than municipal and community partners, was not collected by the program making an assessment of funding overlap and duplication difficult.

Local Festivals survey responses were analyzed by population, using Statistics Canada definitions, to assess differences between small, medium and large population centres and rural areas. A similar analysis was not done for Community Anniversaries, Legacy Fund and unfunded applicants responses. There were too few respondents for results to be generalized.

Core issue 1: continued need for the program

Evaluation questions

• To what extent does BCAH continue to address a demonstrable need?
• To what extent is BCAH relevant and responsive to the needs of Canadians?

Key findings

• The conditions and challenges under which the BCAH program was formed persist today, including: few sources of funding, particularly for small communities and for festivals and events with a citizen engagement focus; the impact of the economy on sources of funding for arts and heritage activities; and an aging arts and culture and heritage infrastructure. Changing demographics have impacted on volunteerism.
• There is consensus among Canadians that arts and cultural experiences contribute to community and individual well-being. Among the benefits they provide are a sense of belonging, increased community pride and social cohesion.
• There was consensus among stakeholders interviewed that BCAH is an effective mechanism to encourage citizen engagement. Recipients view BCAH ’s objective and its outcomes as important for leveraging other sources of funds, offsetting the costs associated with hiring artists, artisans and heritage performers, supporting volunteers and making events accessible to the community. However, compared with Local Festivals and Community Anniversaries, Legacy Fund recipients, were less likely to strongly agree that BCAH was an effective mechanism to increase citizen engagement.
• BCAH was responsive in that it funded all eligible Local Festivals and Community Anniversaries applicants who achieved the minimum required merit score. Between 2011-2012 and 2014-2015, overall, BCAH funded 90% of eligible Local Festivals and Community Anniversaries applicants and 70% of eligible Legacy Fund applicants. Eligible applicants requested $152.5 million. The federal government provided a $68.3 million in Gs&Cs. For the four years covered by the evaluation, the average amount of funding per project relative to the eligible amount requested was 41.3% for Local Festivals and 44.1% for Community Anniversaries projects. The amount paid to Legacy Fund recipients, relative to the requested amount, averaged 74%.
• During the period covered by the evaluation, BCAH did not target funding to more vulnerable communities and projects with greater need for funding support and/or stability or to GC or PCH priority groups. Funding of a small number of large well-established festivals impacted on the amount of funding available for smaller festivals.
• While generally recipients were satisfied with BCAH , they identified areas where the program could be more responsive, including: addressing the fluctuations in funding year-to-year for recurrent festivals; the level of funding received relative to the eligible amount requested; and improving the timeliness and timing of notification of funding decisions (See section 5.3).
• The number of applications for BCAH funding has declined in the last two years of the evaluation period, particularly for the Community Anniversaries and Legacy Fund components. Monitoring and further analysis is required to determine the reasons for the decline.

The evaluation findings illustrated that there is a continuing need for public investment in arts and heritage activities that support citizen engagement. This section discusses the current environment, the factors that contribute to the on-going need for federal investment, the benefits of arts and heritage to Canadians and the program’s responsiveness to recipient needs.

BCAH was created in 2007 to address gaps for festivals and events in Canada, including:

• A lack of federal programming to support non-professional performing arts festivals, community historic events or arts and heritage festivals;
• Gaps in funding for festivals and events with building community, social development or community engagement objectives. Federal funding through regional development agencies, as well as funding from other levels of government, tended to focus on large-scale events of economic significance rather than small-scale celebrations of culture or heritage that sought to build community;
• Funding gaps for smaller communities and for smaller volunteer-driven events. While well-known, large-scale events were found to have greater access to government and corporate funding based on their greater visibility and tourism potential, smaller events, run by volunteers or not-for-profit organizations were not always able to obtain funding from government or corporate sources given their smaller scale; and
• Instability for festivals and events of all sizes due to financial instability, rising costs, and competition for tourist dollars and corporate support.

The Community Historical Anniversaries Legacy Fund component was introduced in 2009-2010 to support larger capital projects, involving the restoration, renovation or transformation of existing buildings and/or exterior spaces that engage Canadians in their communities through the commemoration of major anniversaries of significant local events and/or persons.

The funding gaps which led to federal support for BCAH persist. Small festivals and smaller communities may have a greater need for funding than their larger counterparts.

Factors such as a lack of funding for arts and heritage generally, and more specifically for small festivals and for citizen engagement, the impact of a decline in provincial and territorial (P/T) arts and heritage budgets, aging infrastructure, and changing demographics point to a continued need for the federal government to support arts and heritage activities with the goal of citizen engagement.

Generally P/T funding for arts and heritage projects and activities has remained stagnant or declined in recent years as most P/Ts introduced restraint measures in efforts to balance their budgets. A 2015-2016 analysis of P/T budgets suggests that overall, the pattern of restraint and cutbacks of previous years continues.^{Footnote 5} This financial instability continues to present challenges to festivals and events generally.

Evidence from the literature, recipient surveys, case studies and stakeholder interviews indicated that smaller communities and rural areas, in particular, have suffered from the economic downturn and continue to experience challenges in securing financial and human resources for their arts and heritage projects. As noted in the literature, festivals and projects in rural communities tend to be small, and are less likely to align with and be eligible for funding under P/T tourism investment strategies. These communities “feel they are overlooked by city-centric funding approaches”.^{Footnote 6} Stakeholder interviews and survey results indicated that smaller communities often rely on fewer community partners and the availability of funding is often contingent on the local economic climate and the level of competition for funding among non-profit organizations in their communities.

All PCH officials and provincial and municipal representatives interviewed agreed that there continues to be a need for federal support due to lack of funding generally, but more specifically for small local festivals. Almost all PCH officials pointed to the circumstances leading to federal involvement, confirming that BCAH was created to address the need to fund small festivals in smaller communities. The majority of respondents noted that there is still a need as there is no other federal funding support for this type of festival.

The funding provided through the Local Festivals component is incremental in that most projects would have proceeded, but with a reduced scope, without BCAH funding. None of the Local Festivals recipients surveyed who had received festival funding in one year but not in a previous or subsequent year cancelled their festival in the year that they did not secure BCAH funding. However, only a few proceeded with their festival as planned (14%). Festivals in large urban (21%) and medium (17%) population centres were more likely to proceed as planned compared with small population centres (13%) and rural areas (0%). The majority reduced the scope or modified their activities (66%) and/or reduced their local programming (52%). Elements specifically targeted by BCAH funding, such as attracting, hiring and reimbursing artists, artisans and heritage performers, were reduced. Survey results showed that 7% of applicants who had applied for but never secured BCAH funding cancelled their festival.

Evaluation findings also suggested that some smaller festivals and events in small communities were at greater risk without BCAH funding. Although recipients from rural areas and small population centres also mentioned reducing local artists, they were more likely to mention that the festival could not grow or was in decline, or that the event was at risk. The case studies also suggested a greater need for support from BCAH among small festivals. Interviews conducted as part of the four festival case studies, indicated that the two large and the medium sized festival would have taken place without BCAH funding, albeit with a reduced scope. However, the small festival would not have happened without support from BCAH.

There is a continued need for support to organizations to restore, renovate or transform existing buildings or exterior spaces.

The continued need for support for capital projects that involve the restoration, renovation or transformation of existing building and/or exterior spaces with local community significance was supported by the literature, through interviews and by the case studies which indicated community infrastructure generally, including arts and heritage infrastructure, is suffering from years of neglect and that there is a pent-up need for facilities repair and replacement across Canada. A Federation of Canadian Municipalities report in 2007 noted that there was a $40.2 billion sub-deficit for community, recreational, cultural and social infrastructure, and concluded that many of the facilities needed immediate attention.^{Footnote 7} Many heritage facilities have outgrown their original buildings and are having difficulty managing their collections and exhibiting collections in adequate conditions. Another theme that emerged from the literature is that infrastructure needs have changed as most facilities were built as single purpose entities and are now multi-purpose in nature necessitating retrofits and refurbishments. Small and medium-sized cultural organizations have indicated being cut out of major infrastructure programs because they are “unlikely to be considered a “major” provincial cultural or tourist attraction” and are “generally not able to attract corporate and individual support at the high level of their larger colleagues”.^{Footnote 8},^{Footnote 9}

The need for infrastructure spending has not abated. One provincial representative referenced an infrastructure study done in his province in the 1990’s that showed that at that time the province’s facilities (including cultural infrastructure) were ten years beyond their life expectancy. He estimated that facilities are now 20 years beyond their life expectancy and the situation has not improved. A 2012 evaluation of Parks Canada’s National Cost-Sharing Program for Heritage Places (NCSHP)^{Footnote 10} found the demand for funding exceeded resources in each round of funding, with no more than 60% of eligible recipients receiving funding in any given year.^{Footnote 11}

The case studies of two legacy projects found that there is a greater need for funding support to heritage restoration projects than municipal venue enhancement projects. BCAH funding enabled organizations to leverage funding from other private and public sponsors for the legacy-funded work on a recognized heritage building. While the municipal venue enhancement project would have proceeded without BCAH funding, the heritage restoration project would not have.

There are few sources of funding with the primary objective of building community, social development or citizen engagement.

There are a few other programs at the provincial level with the explicit objective of building community, social development or citizen engagement. British Columbia, Manitoba and Ontario have programs aimed at events that build community spirit, have a clear community or cultural benefit, encourage youth participation and leadership or foster a sense of identity and pride. However, major funding for many programs offered by other levels of government are to promote the economy or tourism, to support professional artists, to cover operating costs or to support specific activities, such as marketing. For example, Alberta, Ontario, Nova Scotia, and Prince Edward Island have festival and event programming which have as their primary objective economic growth and/or increased tourism. Often small communities are not eligible for these funding sources.

Except for Newfoundland and Labrador’s Cultural Events Fund, which provides funding for significant anniversary commemorations, P/T programs related to commemorative anniversaries focus on celebrating, conserving and commemorating local, provincial or national history, and are not usually tied to specific events.

These findings from the scan of programs in other Canadian jurisdictions were supported by interviews with representatives from P/T funding programs and municipal governments. Provincial representatives confirmed that the primary focus of their funding programs for festivals is to stimulate economic development and tourism whereas all municipal representatives commented on the lack of municipal financial resources to support the types of events and projects supported by BCAH . Municipal representatives indicated that municipal support generally has the goal to get people together, to highlight the history of the community, and to attract people from outside as well as from within the community.

Volunteerism in Canada is declining

Many BCAH recipients reported that their volunteer pool is aging and that they have encountered challenges recruiting and retaining volunteers. They identified the need to attract a new generation of volunteers. These challenges are supported by a recent report by Statistics Canada on Giving, Volunteering and Participating, based on data from the 2013 General Social Survey (GSS). This report highlighted the need to encourage volunteering in Canada. The total number of volunteers in 2013 was 4% lower than in 2010, despite an increase in the population aged 15 years and older. The survey also noted that, like the general population, volunteers are aging.^{Footnote 12}

Engaging citizens in their communities through arts and heritage benefits Canadians in many ways, among them promoting social cohesion and a sense of belonging and increased community pride.

Using public involvement in arts and heritage activities as a mechanism for citizen engagement, was based on research showing communities thrive if they have an engaged citizenry that feels a strong sense of identity and attachment to their communities.

The evaluation evidence indicated that arts and heritage experiences generate benefits for individuals and communities and ultimately Canada. Recent literature suggests that bringing people together to work on a common project or objective contributes to “social cohesion” or social capital” and to increasing social trust, reciprocity, a sense of belonging in communities, a sense of place and increased community pride and enhanced community image.^{Footnote 13}, ^{Footnote 14}, ^{Footnote 15}, ^{Footnote 16,} , ^{Footnote 17}

Some provincial funding sources also recognize the contribution of arts and culture to the achievement of social benefits. For instance, Quebec’s Ministry of Culture and Communications notes in its principles and objectives “culture is a vector for democracy, intercultural dialogue and social cohesion” and the Ministry of Community Sport and Cultural Development of British Columbia has as a goal for arts and culture which states “communities are culturally rich and foster sustainable jobs, economic growth and a vibrant social fabric.” ^{Footnote 18}

BCAH ’s objective to engage citizens in their communities through arts and heritage and its outcomes continue to be relevant to Canadians and recipients. However, BCAH ’s objective and outcomes may be less relevant to Legacy Fund recipients.

The 2012 Arts and Heritage in Canada: Access and Availability Survey showed consensus among Canadians on the benefits of arts and culture to communities with large majorities either “strongly” or “somewhat” agreeing that “Arts experiences are a valuable way of bringing together people from different languages and cultural traditions” (93%); “Arts and cultural activities in a community make it a better place to live” (91%); “Exposure to arts and culture is important to individual well-being” (90%);“Arts and culture help us express and define what it means to be Canadian” (87%)“Arts and cultural activities are important to a community’s economic well-being” (86%); and “It’s important to support the arts by volunteering or donating funds or goods” (81%).^{Footnote 19}

BCAH ’s objective also continues to be relevant to recipients. A large majority of survey respondents – local festivals (89%), community anniversaries (89%) and unfunded festivals (93%) – strongly agreed that local festivals, commemorative events or capital projects were effective mechanisms to increase the engagement of citizens in their communities. However, BCAH ’s objective may be less relevant to Legacy Fund recipients, with only 71% strongly agreeing and 29% agreeing to “a moderate extent”.

There were variations among survey responses on the relevance of BCAH funding on the achievement of the intermediate outcomes. The majority of respondents to all four surveys identified BCAH as very important to encouraging local partners to support their project or event and to encouraging citizens to attend events or projects (68%-82%). Local Festivals respondents (funded and unfunded) were the least likely to agree that BCAH funding was very important. Almost all Local Festivals recipients (95%), unfunded festival applicants (97%) and Community Anniversaries recipients (90%) indicated that BCAH funding was very important to providing opportunities for local artists, artisans and heritage performers to perform and present their work. Similarly, the majority (73%-82%) indicated BCAH was very important to encouraging engagement of citizens in their communities through volunteerism. However, Legacy Fund recipients surveyed were significantly less likely than other survey respondents to rate BCAH funding as “very important” to providing opportunities to local artists, artisans or heritage performers to perform and present their work (71%) or to encouraging involvement of citizens in their communities through volunteerism (53%).

Compared with large urban, medium and small population centres, Local Festival survey respondents from rural communities were somewhat more likely to rate BCAH funding as very important to encouraging: involvement of citizens in their communities through volunteerism (82% versus 68%-75%), local partners to support festivals (73% versus 62%-70%) and citizens in their communities to attend festivals (89% versus 81%-83%) and to providing opportunities to local artists, artisans and heritage performers to perform and present their work (90% versus 81%-83%)

There was consensus among the majority of PCH officials, municipal and provincial representatives, recipients and unfunded applicants that BCAH -funded events and projects are effective mechanisms to increase citizen engagement in their communities. However, the majority of PCH officials also identified social and cultural benefits including: connecting and bringing people together; preserving and maintaining arts and culture in a community; creating a sense of belonging which leads to stronger, safer and healthier communities; creating opportunities for citizens to participate, become involved and to give back; promoting a sense of pride and attachment to the community; and providing the opportunity to celebrate ones identity. The majority of P/T and municipal representatives also agreed that BCAH was an effective mechanism for encouraging engagement of community members and organizations, noting that BCAH -funded projects help to increase the dynamism of the community and connect and bring people together. Others saw BCAH as an effective mechanism to educate the community about its history or to present and preserve arts and culture in a community. Legacy Fund recipients indicated that BCAH is an effective mechanism to support renovation or maintenance of a building and were less likely to characterize their project in citizen engagement terms.

BCAH has been responsive to the funding needs of organizations by funding almost all eligible Local Festival and Community Anniversaries applicants who achieve the minimum required merit score. However, the number of applications for BCAH funding, particularly for the Community Anniversaries and Legacy Fund components, decreased in the last two years of the evaluation period.

Between 2011-2012 and 2014-2015 of the 4582 applications received in the four years covered by the evaluation PCH approved 3434 projects (75%) for funding. A proportion of these files were ineligible, did not meet the required merit score or were subsequently withdrawn. BCAH funded 3336 projects, 90% of eligible Community Anniversaries and Local Festivals applicants and approximately 70% of eligible Legacy Fund applicants.^{Footnote 20} Successful applicants requested approximately $152.5 million and the federal government provided a total of $68.3 million in Gs&Cs (Appendix D: Table 2 ). The average amount of funding per project relative to the eligible amount requested for the four years covered by the evaluation was 41.3% for Local Festivals and 44.1% for Community Anniversaries projects. For Legacy Fund projects, the amount paid to the recipient relative to requested amount averaged 74%.^{Footnote 21}

While, the BCAH Local Festival component continued to attract a large number of applications, the program has received fewer applications in the last two years of the evaluation period for all three components. A review of 2015-2016 applications indicated that the number of applications has remained at the levels of the last two years of the evaluation period.

The decrease in applications was more significant for the Community Anniversaries and Legacy Fund components. An analysis of the administrative data indicated that the Community Anniversaries component experienced minor fluctuations in the years between 2008-2009 and 2012-2013, with between 143 and 176 applications per year. The decrease to 56 applications in 2013-2014 and 52 applications in 2014-2015 is a more recent phenomenon.

The Legacy Fund had a higher number of applications following the introduction of the component (82 in 2010-2011 and 71 in 2011-2012 compared with 42 applications in 2012-2013), possibly because Community Anniversaries applicants were encouraged to apply to the Legacy Fund if it was determined to be a more suitable fit. Without further analysis it is difficult to conclude that the fewer applications is a trend or that the lower numbers in recent years is the norm.

Evidence from the document review, interviews and recipient surveys suggested that a combination of factors may be contributing to the decline in applications. Program staff indicated that the decline in applicants to the Local Festivals component may be due to a normal maturing of the program as ineligible festivals or festivals that have been found ineligible once are now unlikely to reapply and as some festivals have grown, they have become venues for more professional, non-local artists and, therefore, no longer meet the program’s eligibility criteria. The decline in applicants to the Community Anniversaries and Legacy Fund component may be due to the nature of the projects, as the eligibility requirements require projects to be linked to a 100th anniversary or greater in increments of 25 years (e.g. 125th, 150th), such as the establishments of towns; therefore, trends in applications are influenced by “fluctuations in history”.

A few PCH officials indicated that BCAH is not reaching certain groups such as recent immigrants and Aboriginal communities. In the case of Aboriginal groups, it may be difficult for them to demonstrate that their event marks a 100th anniversary or greater. As explained by one PCH respondent, “there may be things that could have been celebrated in the Aboriginal side but BCAH requires dates and evidence to be eligible. Aboriginal people have more of an oral tradition and not a written one. For example, the Commemoration Policy Directive in the Indian Residential Schools Settlement Agreement, for funding to commemorative projects proposed by Aboriginal people between 2011 and 2013, did not require reference to a particular year or anniversary. In a section entitled “Public memory: dialogue, the arts, and commemoration” the Truth and Reconciliation Commission final report (2015) refers to commemoration as “repairing and revitalizing individual, family, and community memory.”^{Footnote 22} A few PCH respondents also suggested that since Pow Wows are the main form of celebration for Aboriginal people and as they often have a competitive component, they are not always eligible for BCAH funding.

The decline in applications for the Community Anniversaries and Legacy Fund components in the context of the state of arts and heritage infrastructure in Canada suggests that PCH regional offices could engage with other levels of government in their regions to identify and prioritize potential sites for investment. The majority of PCH officials interviewed suggested that the program could be more strategic in its outreach to organizations and groups that may not be aware of BCAH such as youth and immigrant or Indigenous communities or for certain components. One external expert suggested that the tri-level meetings^{Footnote 23} could be a forum for information-sharing and the prioritization and coordination of infrastructure projects. However, any actions taken by the program to increase uptake requires a balancing of available funds with the demand for funding.

Funding to large well-established festivals has an impact on the amount of funding available for smaller festivals.

One of the gaps leading to the creation of BCAH was the need for funding for smaller communities and for smaller volunteer-driven events. In interviews, the majority of PCH officials highlighted the continued gap in funding between cities and rural areas, big and small festivals and big and small communities. A few Local Festivals recipients surveyed suggested that the distribution of BCAH funding should favor smaller municipalities or rural areas because smaller rural organizations have difficulty raising funds for their festivals while large festivals are better able to acquire corporate support.

One impact of the approach to fund all eligible Local Festivals and Community Anniversaries applicants that achieve the minimum merit score, highlighted by some program staff, is that the program does not target funding to support projects that are more vulnerable or require funding support and/or stability or specific priority groups. Some additional support is provided to small festivals insofar as those that request small amounts of funding are automatically topped-up to $2000. However, small festivals or festivals in small communities compete for limited funding with larger, successful and well-established festivals. Festivals with large total project expenses generally have higher eligible expenses and request and receive more funding. Administrative data on the distribution of funding for BCAH festivals showed that the smallest 914 projects (defined as having total project expenses less than $50,000) represented 32% of all projects but received less funding ($3.7 million) combined than the largest 95 projects (defined as having project expenses greater than $1 million), representing 3% of total projects, which received about $6.5 million, or 16% of the total Festivals funding (Appendix D: Table 4). For festivals with large project expenses, BCAH funding often represented a small percentage of their total project budgets.

The administrative data also showed that many of these large festivals have had a lengthy funding history with BCAH . Between 2011-2012 and 2014-2015, of the 95 large festivals 45 were recurrent. Of these 45 festivals, 33% received funding for seven years and another 16% received funding for six years. A similar analysis for small festivals found that, compared with large festivals, a smaller proportion of small festivals received funding for an extended period of time. Between 2011-2012 and 2014-2015, of the 914 small festivals, 457 were recurrent. Of these 457 festivals, 12% received funding for seven years and another 14% received funding for six years.

Survey results suggest that organizations in urban and medium population centres may rely less on Local Festivals funding than rural areas and small population centres. Fewer respondents from large urban (70%) and medium (66%) population centres indicated that BCAH funding enabled them to undertake their festivals to a great extent compared with respondents from rural areas (80%). Respondents from rural areas (28%) and small population centres (9%) were also more likely to indicate that their festival would not have taken place without BCAH funding compared with respondents from large urban (4%) or medium (5%) population centres.

These findings suggest that consideration be given to examining the funding amounts to large well-established festivals with a view to “graduating” them from BCAH , so that funds can be redirected to address the needs of smaller festivals or smaller communities, or potentially to support GC diversity objectives and PCH priority groups including youth and ethno-cultural and Indigenous Canadians.

Emerging trends

Although several emerging trends were identified, each represents a unique response and no overall trend emerged. According to the experience of an international expert on festivals, consulted for this evaluation, the dominant focus of many governments from a global perspective in the past twenty years has been attempts to measure how festivals contribute economically to communities. More recently, he has observed that the focus has broadened slightly to include the goals of community-building and place-making and to target children, hard-to-reach groups and minorities. There is also a trend toward festivals and events that make a concerted effort to maximize attendance through explicit audience development strategies, or active audience involvement in policy direction and management.

P/T and municipal representatives noted the increased popularity of multicultural festivals as an important venue to celebrate a group’s a cultural identity; more engagement with new artists and with Indigenous artists; the emergence of a new type of festival put on by promoters^{Footnote 24} – “bringing artists from around the world and they take over the valley; people camp (40,000). It is very commercial…but in the landscape of BC it is a big change…Even more established festivals are saying it is having an impact on them. It is not direct competition but it certainly is targeting our younger people”; an increase in the number of festivals occurring in small communities, often as a way of defining themselves as a destination; new technologies which are redefining festivals; and an increased appetite for festivals with existing festivals wanting to grow and to feature higher quality professional performers or artists.

The literature supports the observations of key informants that small communities, in their efforts to diversify their economic base are using festivals as a way of defining themselves as a tourist destination.^{Footnote 25},^{Footnote 26} ,^{Footnote 27} As noted by one author, arts and cultural activities can play an important role in the process of community adaptation, development and reinvention: “As rural and small communities adjust to dynamically changing situations, and position (or re-position) themselves for the future, cultural activities and creative enterprises are increasingly considered as an important element in the sustainability of the community as well as a key in its transition.”^{Footnote 28}

BCAH does not explicitly target or collect data on the engagement of specific groups such as youth or Indigenous or ethno-cultural groups. However, in describing their festivals about 25% (96/379) reported their festival as being multi-cultural/ intercultural (23), celebrating Francophone, Acadian, First Nations or Métis heritage (37) or celebrating their ethno-cultural heritage (36). 18 ethno-cultural groups were represented in these 36 responses.

The literature suggests that multicultural festivals play a significant role as “one instrument in the development and inculcation of successful multiculturalism.”^{Footnote 29} The international expert interviewed noted that “provincial or municipal festivals, and cities are finding the need to review their identity in a world … which is changing significantly in a demographic point of view, largely from immigration. Cities that were one kind of place 25 years ago are a very different place now, and are using festivals and events as a means of negotiating that change, along with many other tools, to shift the image to be in direct relationship with the reality of the place.”

Core issue 2: alignment with government priorities

Evaluation question

To what extent is BCAH aligned with:

• Federal government priorities; and
• PCH strategic outcomes?

Key findings

• BCAH supported GC priorities with respect to supporting and investing in vibrant communities and celebrating/supporting our arts and cultural communities, as expressed in successive federal budgets.
• BCAH aligned with the Whole-of-Government Framework, within the Social Affairs spending area, which outlines support for initiatives that result in a vibrant Canadian culture and heritage.
• The program aligned with the Department of Canadian Heritage’s Strategic Outcome 2: “Canadians share, express and appreciate their Canadian identity” and two departmental priorities – bringing Canadians together: investing in our communities and celebrating our history and heritage.

Government of Canada priorities

BCAH supported GC priorities with respect to supporting/investing in vibrant communities and celebrating/supporting our arts and cultural communities, as expressed in successive federal budgets (2011, 2012, 2013 and 2014).

BCAH also aligned with the Whole-of-Government Framework, within the Social Affairs spending area, resulting in a vibrant Canadian culture and heritage.^{Footnote 30}

BCAH has demonstrated flexibility in supporting GC priorities. In response to the 2013 Budget, the BCAH ’s terms and conditions and application guidelines were modified so that effective January 2014 – March 31, 2017, eligible Community Anniversaries Programming and Legacy Fund projects can include commemoration of 75th anniversary events directly related to Canadian participation in World War II. In 2014 the program funded projects that commemorated milestones on The Road to 2017, thereby aligning some of its projects with the GC priorities and the PCH priority – Celebrating our History and Heritage.

Strategic priorities and outcomes

The PCH DPR s and RPPs associated with the period covered by the evaluation indicate that the objectives and outcomes of the program were closely aligned with PCH’s strategic outcomes. BCAH is a subprogram under Program: Engagement and Community Participation. This program activity aims to ensure that Canadians are engaged and have the opportunity to participate in the civil, social and cultural aspects of life in Canada and in their communities. This is accomplished through, among other things, funding programs and initiatives that support the efforts of communities to build stronger citizen engagement and social inclusion through the performing and visual arts; and the expression and celebration and preservation of local heritage. This program activity aims to provide social benefits in contributing to the preservation of the history and identity of Canada’s diverse communities, while offering a way for traditions and identities to evolve over time. The program activity supports the Department’s mandate to strengthen Canadian identity and values and build attachment to Canada.

According to the 2014-2015 Departmental Program Activity Architecture (PAA) BCAH aligned with the PCH Strategic Outcome 2: “Canadians share, express and appreciate their Canadian identity”. This speaks to the importance of arts and culture in helping build stronger communities by bringing people together through shared artistic experiences.^{Footnote 31}

BCAH also supported two Departmental priorities: Celebrating our History and Heritage and Bringing Canadians Together: Investing in Our Communities. As noted in the 2014-2015 RPP, the priority Bringing Canadians Together supports activities that are grounded in local and community realities so as to encourage the sharing of our diverse cultural expressions and understanding of our history and heritage. This helps to connect communities, contribute to healthy and vibrant communities and develop a strong sense of Canadian identity.

Core issue 3: alignment with federal roles and responsibilities

Evaluation question

To what extent is BCAH aligned with federal roles and responsibilities?

Key findings

• The delivery of the BCAH is an appropriate role for the federal government, notably due to its interests in engaging citizens in civic life, in fostering pride and attachment and in promoting Canada’s arts and heritage sectors, as well as due to the lack of alternate sources of funding, generally and in particular for events and projects with a citizen engagement objective.

Ensuring that Canadians have continued access to arts and heritage experiences and are engaged in their communities is an appropriate role for the federal government.

BCAH ’s objectives support the Minister of Canadian Heritage in exercising her powers, duties and functions relating to Canadian identity and values, cultural development and heritage. In exercising the powers and performing the duties and functions assigned by section 4, the Minister shall (a) initiate, recommend, coordinate, implement and promote national policies, projects and programs with respect to Canadian identity and values, cultural development and heritage (section 5(a)).^{Footnote 32}

BCAH supports the Departmental Mission and its roles and responsibilities, as derived from its legislative mandate by promoting an environment in which all Canadians take full advantage of dynamic cultural experiences, celebrate their history and heritage and participate in building creative communities. BCAH ’s focus on citizen participation, actively contributes to the mandate of the Citizenship, Heritage and Regions Sector which encourages “Canadians to get engaged in the civic life of their community and of Canada, while respecting and recognizing the multiplicity and the complexity of their attachments, to broaden their knowledge of Canada and its history and to acquire experiences that deepen their identity as Canadians.”

Canadians believe that the federal government, as well as other levels of government, have a responsibility to ensure access to programs and activities that promote arts and heritage across Canada. The Arts and Heritage Access and Accessibility survey (2012) shows that a large majority of Canadians agreed that governments should: help ‘protect and preserve Canada’s heritage (95%); promote awareness of Canadian arts and cultural events and activities (91%); provide support for arts and culture in Canada (90%); partner ‘with others to ensure that there are enough arts and cultural facilities to serve the public (89%); and provide “incentives to promote private sector support for arts and culture” (85%).^{Footnote 33}

The vast majority of interview respondents, notably both PCH staff and external stakeholders, agreed that investing in building stronger citizen engagement in communities through arts and culture is a legitimate federal government role. A few noted that all levels of government should be involved. PCH stakeholders supported a federal role on the basis that it promoted Canadian arts and heritage, encouraged a Canadian identity; promoted attachment and pride to be Canadian; promoted inclusivity^{Footnote 34} and provided the credibility necessary for organizations to attract other sources of funds.

All provincial and municipal representatives, unfunded applicants and funded recipients supported a federal role. The majority cited the lack of available funding for local festivals, commemorative anniversaries and legacy projects as the primary reason for a federal role. Recipients mentioned the importance of federal funding as leverage to secure funding from other sources. A few recipients also noted that BCAH ’s focus on culture and history with the goal of promoting a national identity is an appropriate role for the federal government.

While community development is a provincial responsibility, stakeholders perceive the federal funding to be complementary to the other sources of funding. BCAH ’s Gs&Cs are a key policy lever for the federal government to support citizen engagement in their communities while promoting arts and heritage. BCAH funding acts as catalyst to encourage organisations to forge financial partnerships and to stimulate citizen participation in arts and heritage activities.

Key informants identified other potential roles, aside from funder, that could be played by the federal government. A majority of PCH officials (8/9) interviewed noted that building capacity in organizations with lower capacity is another role that the federal government could play. Examples included advising recipients how to do budgets, develop their citizen engagement strategies such as volunteer recruitment or to find new sources of community funding; providing seminars on preparing grant applications, noting that some of their other funders provide this service; and facilitating the sharing of best practices among recipients. A few PCH officials noted that this is already done but is not recognized.

From the perspective of an international expert in the field of festivals and events, the federal government should play a complementary role, to attempt to innovate through filling gaps such as supporting initiatives which are new and not yet in the position to access other funding sources, that address children, or diversity, or hard-to-reach audiences, that create a cross-country learning environment, are of significance due to size or scale that are out of the ability of local funders to support, or which have national significance. The role of a federal agency should be strategic – to stimulate or incentivize rather than replace funding that should come from the more local stream; it could be contrary to encouraging local authorities to take festivals and events more seriously if federal funding is there to replace, rather than encourage or increase sustainability of local funding. Another potential role for the federal government is to encourage learning, through the collection of evidence-based examples and knowledge transfer, which could include developing a comprehensive web-site with materials that are gleaned, best practices, case studies and stories from individual festivals. Finally, the federal government could have a role in identifying skill shortages; soliciting sponsorships, marketing; building audiences; or advising on how to involve the local community more in managing the festival.

En raison de l’exercice de réduction du déficit mené par le gouvernement du Canada dans les dernières années, les responsables du programme DCAP ont simplifié les processus et modifié les activités pour améliorer l’efficacité du programme. Les activités proposées par les informateurs clés relativement au rôle éventuel en matière de renforcement des capacités nécessiteraient de déroger au mandat et aux activités actuels du programme. Si l’on envisageait ces responsabilités supplémentaires, il faudrait réaliser une analyse de faisabilité, plus particulièrement en ce qui touche la capacité d’assumer un rôle élargi et de répondre aux besoins connexes, étant donné que l’évaluation a révélé l’atteinte des objectifs du programme.

In response to the GC deficit reduction exercise in recent years, the BCAH program has streamlined processes and modified activities to operate more efficiently. The proposed activities associated with the potential capacity-building role, as suggested by key informants, would represent a departure from the current mandate and activities of the program. Any consideration of these additional responsibilities would require an analysis of feasibility, particularly the capacity to assume an enhanced role and need, given that the evaluation had demonstrated that the program is achieving its outcomes.

Core issue 4: achievement of expected outcomes

Achievement of immediate outcomes

Evaluation question

To what extent BCAH achieve its immediate outcome?

• Local organizations receive financial resources to carry out local festivals, community anniversaries or legacy projects in their communities.

Key findings

• Between 2011-2012 and 2014-2015, 3336 BCAH projects received approximately $68.3 million in PCH funding.
• BCAH funding has helped recipients offset the costs associated with engaging artists, artisans and heritage performers and with supporting volunteers by providing a sometimes vital amount of funding, particularly for Legacy Fund recipients and recipients from rural areas. By being able to offset these costs, recipients were able to make their event more accessible to their community.
• BCAH funding was important for all recipients but was more important to Legacy Fund recipients and recipients in rural areas.
• Fluctuations in funding from year to year has created challenges for some recurrent festivals, particularly with respect to their ability to plan ahead and to leverage BCAH funding to secure other sources of funds.

Between 2011-2012 and 2014-2015, 3336 BCAH projects received approximately $68.3 million in federal funding to support local artists, artisans and heritage performers and volunteers.

BCAH funding is directed to supporting local artists, artisans and heritage performers and volunteers and include costs (excluding salaries) of recruiting, training, supporting and recognizing local volunteers, fees and expenses for local artists, artisans and performers of local heritage activities, fees related to exhibiting artwork by local artists and artisans and copyright and permits, costs of publicity aimed at the local population, presentation and logistical expenses (e.g., traffic barriers, portable toilets, garbage bins), venue rental and set-up costs, the cost of insurance for eligible activities and expenses related to financial audits and environmental assessments when required by the Department.

During the period covered by the evaluation, 3336 BCAH events and projects received approximately $68.3 million through Gs&Cs. The largest proportion of funding for the Community Anniversaries (81.5%) and Local Festivals (94.3%) components was in the form of grants. However Legacy Fund projects, due to their larger funding amounts, were mainly funded through contribution agreements (71.1%). The Local Festivals component accounted for 68% of BCAH funding, with the Community Anniversaries component accounting for 22.1% and Legacy Fund projects for 15.8% (Appendix D: Table 4).

Administrative data indicated that for the Legacy Fund in particular, the average total amount paid to the recipient as a proportion of their total project expenses (26.2%), or as a proportion of the total amount requested in their applications (74.1%), can be significant.

Funding to Local Festivals and Community Anniversaries recipients is based on a policy decision to fund all eligible applicants who achieve a passing merit score. The amount of funding received by each applicant depends on a number of factors which feed into a funding formula, including population, the number of applications received, the eligible amounts requested by applicants and their merit scores. Funding to Legacy Fund recipients is based on merit. Over the four year evaluation period about 90% of eligible Local Festivals and Community Anniversaries applicants were approved for funding whereas 70% of eligible Legacy Fund applicants were approved. As a result of the larger number of eligible applicants, the amounts paid to recipients as a proportion of total project expenses was less (averaging 6.3% for the Local Festivals component and 13.1% for the Community Anniversaries component) than for the Legacy Fund component (26.2%). The amount paid to the recipient as a proportion of the eligible amount requested in their applications averaged 41.3% for the Local Festivals component and 44.1% for the Community Anniversaries component over the same period. For Legacy Fund projects, the amount paid to the recipient relative to requested amount averaged 74%.^{Footnote 35}

The evaluation evidence indicated that BCAH funding was important to all recipients but was more important to Legacy Fund recipients and recipients in rural areas.

Recipient interviews and survey results indicated that BCAH funding had a significant impact on the ability of the majority of recipients to undertake their events and projects; however funding had a greater impact on the ability of the Legacy Fund recipients and recipients in rural areas. Legacy Fund (91%), Community Anniversaries (78%) and Local Festivals recipients (71%) indicated that BCAH funding enabled them to undertake their projects to “a great extent”. A few Local Festivals respondents (6%) and Community Anniversaries respondents (3%) indicated that BCAH had a small impact on their ability to undertake their project. Local Festivals recipients from rural areas (80%) were more likely to indicate that BCAH funding enabled them to undertake their projects to “a great extent” compared with festival respondents generally (71%). Legacy Fund recipients and rural recipients of Local Festivals funding were also more likely to indicate in their qualitative comments that their project could not have occurred without BCAH funding. Legacy Fund recipients interviewed and case studies also found that since BCAH was their major funder, their legacy projects would not have occurred without BCAH funding.

Further analysis of the Local Festivals survey results showed that respondents^{Footnote 36} who indicated that BCAH had a significant impact on their ability to undertake their event were generally more satisfied with the amount of funding received from BCAH relative to total project expenses (generally greater than 20%) and that BCAH was their primary or most important partner. Those who indicated a moderate or small impact (27.8%) were more likely to indicate that the funding received from BCAH was a small or negligible proportion of their total project expenses (generally less than 15%). A few recurrent festival recipients commented that their funding had decreased over the years, with a corresponding decrease in the impact of BCAH funding.

Some recurrent festivals were dissatisfied with the fluctuations in funding from year-to-year.

The evaluation evidence indicated that fluctuations in funding from year to year created challenges for some recurrent festivals, particularly with respect to their ability to plan ahead and to leverage BCAH funding to secure other sources of funds. Program documentation showed that for each intake, a proportion of Local Festivals recipients received lower funding amounts than they had received for their previous festival. This may be one reason why 33% of Local Festival survey respondents expressed some level of dissatisfaction with the level of funding provided compared to what was requested indicating that they were somewhat dissatisfied (23.4%) or very dissatisfied (9.6%). Of the 21 Local Festivals recipients surveyed who provided additional comments about their funding levels, eight indicated that their funding allocation had decreased from previous years; six commented on funding instability year to year; and seven commented that they had received less than requested. An additional seven survey respondents commented that they would like an explanation on how allocations were calculated.

Achievement of intermediate outcomes

Evaluation question

To what extent did the BCAH program achieve its expected intermediate outcomes?

• Local partners within the community provide support to funded local festival, community anniversaries and legacy projects in their communities;
• Local citizens are provided with opportunities to engage in their communities through local arts and heritage;
• Local artists, artisans and/or heritage performers are provided with opportunities to engage in their communities through local arts and heritage; and
• Local citizens have opportunities to be exposed to local arts and heritage.

Key findings

• Funded organizations contributed to the achievement of BCAH ’s intermediate outcomes. Projects successfully secured local partners; obtained cash and in-kind support from municipal and community partners, as well as non-local partners; engaged artists, artisans and heritage performers and volunteers; and provided opportunities for community participation in local arts and heritage activities. Overall, the program exceeded its targets for its intermediate outcomes in each of the years of the evaluation period.
• Recipients leveraged BCAH funding to secure cash and in-kind support from other sources. Over the four year period of the evaluation, for every $1 of BCAH funding, funded organizations secured $5 in cash and in-kind support from municipal and community partners. Leveraging was more important for Legacy Fund recipients.
• The program’s requirement for “local” artists, artisan and heritage performers created challenges, particularly for smaller communities, who may have a limited number of artists, artisans and heritage performers in their communities.

Organizations were successful at securing local partners. For the two years for which data is available (2013-2014 and 2014-2015), BCAH -funded projects secured 38,200 partners.

The number of local partners was introduced in 2013-2014 as an indicator of the level of community commitment to the project or event. ^{Footnote 37} Administrative data for 2013-2014 and 2014-2015 showed that during these two years, Local Festivals and Community Anniversary components, engaged a significant number of community partners in their events averaging 32 and 24 per project respectively. Legacy Fund projects had fewer partners averaging 14 community partners per project for the two years (Appendix D: Table 5).

“Interestingly, funds received from a governmental source often generate funds from other sources. It seems to underline a level of credibility that other sponsors and donors are looking for. It is not the amount that matters, it is the idea that a government agency thinks the event is worthwhile.” (Recipient)

“As a new festival, we needed support of provincial and national funding bodies to help legitimize the event for the private sector that we were looking for in-kind support from.” (Recipient)

Survey results were comparable to the results from the administrative data. The majority of Local Festivals (92%), Anniversaries (82%) and Legacy Fund (85.7%) recipients surveyed indicated that they were very or moderately successful in obtaining support from local partners. Community Anniversaries (31%) and Local Festivals (39%) recipients reported having 20 or more partners. The majority of unfunded applicants surveyed were also successful at securing partners with 32% reporting that they had more than 20 partners. However, fewer (5%) Legacy Fund recipients reported more than 20 partners.

Recipients leveraged BCAH funding to secure cash and in-kind support from other sources. Leveraging was more important for Legacy Fund recipients.

BCAH funding constituted a small proportion of total project expenses, particularly in the case of Local Festivals and Community Anniversaries components. Federal funding was complemented with funding from other levels of government and funding from the not-for-profit sector, private sector and donations.

Half of PCH officials interviewed noted that BCAH encouraged recipients to work with local partners and that money provided by BCAH helped recipients leverage other sources of funds. The majority of recipients surveyed reported that their municipal governments and local community partners as well as P/T governments and non-local partners, were more likely to support their project once they had secured BCAH funding. Over 40% of unsuccessful applicants surveyed indicated that failure to secure BCAH funding adversely impacted their ability to obtain financial and in-kind support from other sources.

Organizations were successful in securing cash and in-kind support from municipal governments and community partners. Between 2011-2012 and 2014-2015, BCAH -funded projects secured a total of $298.6 million in cash and in-kind support from municipal and community partners for their events and projects.

Over the four year period of the evaluation, for every $1 of BCAH funding, funded organizations secured $5 in cash and in-kind support from municipal and community partners.^{Footnote 38}

Municipal cash and in-kind support

Application guidelines specify that funding recipients are required to provide written confirmation of financial or other tangible support from the municipal government or equivalent authority for their event or project. Between 2011-2012 and 2014-2015, BCAH -funded projects secured a total of $79,979,733 in cash and in-kind support from municipal governments ($47.9 million in cash support and another $32.1 million in-kind support), averaging $17.9 thousand in cash support per project and $11.6 thousand in in-kind support from municipalities (Appendix D: Table 6).

• The Local Festivals recipients secured a total of $34.6 million in cash support and $27.6 million in in-kind support. The average cash support per project for this period was $14,445 and $11,122 for in-kind.
• The Community Anniversaries recipients secured a total of $9.6 million in cash support and $3.6 million in in-kind support, averaging $39,189 and $14,944 per project, respectively. The decline in the total cash and in-kind support in 2013-2014 and 2014-2015 reflected the decrease in the number of funded projects.
• The Legacy Fund recipients secured a total of $3.7 million cash support and $866,780 in- kind support for an average per project of $99,286 and $27,087 respectively. It should be noted, that incomplete data was available for the Legacy Fund component for 2014-2015.

Community cash and in-kind support

Between 2011-2012 and 2014-2015, BCAH -funded projects secured a total of $218.6 million in cash and in-kind support ($92.3 million in cash support and another $126.3 million in in-kind support) from community partners, averaging $32,663 for cash support and $44,434 for in-kind support per project (Appendix D: Table 7).

• The Local Festivals component secured a total of $79.9 million in cash support and $119.6 million in in-kind support. The average per project cash support for this period was $31,520 and $46,664 for in-kind.
• The Community Anniversaries component secured a total of $6.9 million in cash support and $5.3 million in in-kind support, averaging $27,485 and $21,286 per project respectively. Cash and in-kind support declined significantly in 2013-2014 and 2014-2015, reflecting the decrease in the number of funded projects.
• The Legacy Fund component secured a total of $5.5 million in cash support and $1.5 million in in-kind support for a per project average of $137,620 and $44,577 respectively.

Aside from the data on cash and in-kind support from municipal and community partners, the program does not collect data on cash or in-kind resources secured from other sources, so it is difficult to obtain a clear picture of funding support to BCAH -funded projects. However, evidence from surveys and interviews with recipients indicated that in addition to municipal and community partners, recipients also leveraged BCAH funding to secure cash and in-kind support from various other sources including other federal departments, P/T governments and non-local sources.

• Community Anniversaries recipients secured cash support from local individuals (44%), community associations (41%) and P/T governments (41%). Other sources of in-kind support were local community associations (58%) and local individuals (52%).
• Local Festivals recipients secured cash support from P/T governments (60%), local community associations (41%) and local individuals (43%). In-kind support was secured from local community associations (59%). Unfunded festivals had a similar funding profile.
• Legacy Fund recipients secured cash support from local individuals (62%), community associations (52%), P/Ts (47%). Their main sources of in-kind support were community associations (57%) and historical societies (52%).

“We are located in a very small town that lost its major industry about 5 years ago resulting in less businesses and community downsizing. So many local organizations, so few businesses to support them all.” (Recipient)

Local Festivals survey results indicated that, compared with medium and small population centres and rural areas, organizations from large urban population centres were more likely to report that they secured cash support from other federal government departments (37% versus 11% to 25%), P/Ts (68% versus 53%-60%) and municipal governments (83% versus 51% to 72%).

Challenges associated with securing partners and cash and in-kind support were consistent across components and included the volatility of funding, availability of new cash support or the ability to sustain or increase support from local partners year to year as a result of the local economic climate and competing demands for support on community organizations. About a quarter of Local Festivals recipients who identified challenges noted that it was more difficult to secure cash support than in-kind support. A few recipients noted that it was difficult for arts and history-related activities to compete for support with other activities, such as sports.A few recipients also expressed concern about the capacity of volunteers, specifically the time and skills to pursue additional partners and the time and effort required to work with partners.

Comments from survey respondents suggested that challenges for rural and small communities in securing cash and in-kind support are often associated with having a smaller pool of businesses, particularly large businesses, to approach. There is also greater competition for support as the few businesses that exist are usually solicited for every community event.

Local citizens were provided opportunities to engage in their communities through local arts and heritage. The program’s targets of 149 volunteers and 3,954 volunteer hours per project were exceeded in each of the years covered by the evaluation.

The program’s administrative data indicated that the program’s targets of 149 volunteers and 3,954 volunteer hours^{Footnote 39} per project were exceeded (Appendix D: Table 9 and Table 10).

• Between 2011-2012 and 2014-2015, the Local Festivals component, attracted 424,894 volunteers averaging 161 volunteers per project. The total number of volunteer hours was 12,010,371 for an average of 4,563 per project.^{Footnote 40} The average number of volunteers and the average number of volunteer hours increased steadily between 2011-2012 and 2014-2015. Although the total number of volunteers and volunteer hours declined in 2014-2015, the average number of volunteers and volunteer hours increased.
• The Anniversaries component, attracted 41,262 volunteers between 2011-2012 and 2014-2015, averaging 154 volunteers per project. The total number of volunteer hours was 769,406 or 2,903 per project.
• The Legacy Fund component attracted a total of 2,930 volunteers for an average of 65 volunteers per project and 63,591 volunteer hours for an average of 1,413 volunteer hours per project.

It should be noted that not all Legacy Projects which were approved in 2014-2015 had submitted their final reports at the time of the evaluation.

The majority of recipients interviewed indicated that they were successful in securing local volunteer support and highlighted the importance of volunteers to the success of their projects. Survey results showed that Legacy Fund recipients (62%) indicated less success engaging local volunteer support than Local Festivals (72%) and Community Anniversaries (70%) recipients. Large urban population centres (78%) had greater success at engaging volunteers compared with medium (69%) and small population centres (70%) and rural areas (70%). Unfunded festivals (46%) were less successful in obtaining local volunteer support than BCAH -funded projects.

In terms of successes and challenges, a key success highlighted in comments from some (21/153) Local Festivals recipients was the participation of youth as volunteers. Organizations experienced challenges associated with an aging volunteer pool and the need to recruit and retain volunteers. For the Local Festivals survey, 31/40 or 78% of recipients who provided comments noted recruitment and retention as an issue. According to some PCH officials and recipients, volunteer burnout is particularly acute in small population centres and rural areas, where the same core group of volunteers are called upon to volunteer for all community events. Another challenge identified by a few festival recipients was the need for experienced volunteers to assume planning or chairperson responsibilities or a qualified coordinator or administrator, preferably a paid position, to manage the festival. This was supported by provincial representatives interviewed who pointed out that organizers often lack the capacity or expertise to plan, to adjust their activities and budgets as changes occur to the project, to find matching cash contributions or, in the case of infrastructure projects, to set priorities for renovation.

BCAH projects provided local artists, artisans and/or heritage performers with opportunities to engage in their communities. The program exceeded its target of 125 local artists, artisans and/or heritage performers per project in all years of the evaluation period.

The evaluation evidence from program documentation, surveys, interviews and case studies indicated that BCAH projects provided opportunities for local artists, artisans and heritage performers to be involved in their community through festivals, events and projects. Between 2011-2012 and 2014-2015, program data indicated that 451,686 artists, artisans and heritage performers were provided opportunities to engage in their communities through local arts and heritage events. The average per project ranged from a low of 146 in 2013-2014 to a high of 167 in 2014-2015. The average for the period of the evaluation was 158 artists, artisans and heritage performers per project (Appendix D: Table 11).

As one of its merit criteria, BCAH assesses projects on the basis of the number of local arts and heritage artists or artisans and heritage performers who will perform in the event. Survey results indicated that for the majority of the Community Anniversaries (88%) and Local Festivals (78%) projects more than 50% of the artists, artisans, or heritage performers who participated in the event were local. The results were significantly lower (48%) for the Legacy Fund projects. As one Legacy Fund recipient commented, it is difficult to engage a local artist during a restoration project. Also, compared with Local Festivals recipients, fewer unfunded Local Festivals applicants reported more than 50% local content.

The majority of Community Anniversaries, Local Festivals recipients and unfunded applicants surveyed strongly agreed that local artists, artisans and heritage performers added value to their event or project (89% to 90%); benefited from participating in the event or project (79%-88%) and that the event or project encouraged collaboration between artists, artisans and heritage performers and the local community (76%). Legacy Fund recipients were significantly less likely (67%, 48% and 38% respectively) to strongly agree.

The recipient surveys, interviews with PCHofficials and recipients, as well as the case studies, provided insights on the challenges associated with providing artists, artisans and heritage performers with opportunities to engage in their communities. The most frequently reported challenge from recipient surveys and interviews was the availability of “local” talent. Half (4/8) of Community Anniversaries recipients and 20% (26/132) of Local Festivals recipients surveyed commented that the distance restriction to 100 km creates difficulties with respect to the availability of talent within this range. A case study of a Legacy Fund project found that Legacy Fund projects require fewer local artists and artisans as recognized by efforts by staff to ensure Legacy Fund applicants were not penalized.

Survey results, case studies and interviews with PCH officials and recipients provided evidence that the definition of “local” is particularly problematic for small and remote communities. In the surveys, the issue was raised most frequently by Local Festivals respondents from small communities (16/26). In interviews, the majority of PCH officials (5/6) and 4/6 recipients (Festivals (2/4) and Community ‘Anniversaries (2/2)) noted that smaller communities have a limited number of artists and artisans and heritage performers so the quest for local artists can be difficult year after year. Also, noted by a few was that audience numbers could be adversely impacted if the same talent were presented year after year and the event would be more likely to succeed if local artists were complemented by outside talent.

Some recipients requested that the definition of “local” be expanded. Anecdotal evidence from interviews with PCH staff indicated that staff may already be exercising some flexibility when advising recipients as to what constitutes “local”. However, consideration should be given to clarifying the term “local” in program documentation.

BCAH provided opportunities for local citizens to be exposed to local arts and heritage. BCAH exceeded its annual target of 22,343 visitors/attendees per project.^{Footnote 41}

Between 2011-2012 and 2014-2015, BCAH -funded projects totaled 82,493,304 visitors, for an average of 28,436 visitors per project (Appendix D: Table 12). ^{Footnote 42} Overall, attendance increased year to year. However, caution should be exercised in interpreting the attendance results as recipients are only required to provide an estimate of the attendance at their event. The program also collects data on attendance at arts and heritage activities associated with the Local Festivals and Community Anniversaries projects. Between 2011-2012 and 2014-2015, there were 50,204,850 visitors to arts and heritage activities for an average of 19,031 attendees per project (Appendix D: Table 13).

The vast majority of recipients interviewed (11/12) indicated that their events were accessible to the community because they were free or low cost. This was supported by survey results; the majority of Community Anniversaries (62%) and Local Festivals (68%) recipients surveyed indicated that their events were inclusive and accessible to all members of the local community by being free or at reduced cost. A few Local Festival recipients offered free admission to specific populations including students, clients of social services and the elderly. Survey results indicated that in addition to reduced ticket and passport pricing, the majority of Local Festivals and Community Anniversaries events were also accessible (to a great extent or moderate extent) through the hours open to the public, location of the event/project and parking availability; however, were less accessible to persons with disabilities and through the availability of special transportation and shuttle services. Legacy projects were accessible and visible to the local community based on hours of operation, location of the building or exterior space, accessibility to persons with disabilities and parking availability.

Achievement of the ultimate outcome

Evaluation questions

To what extent did the BCAH program achieve its expected ultimate outcome?

• Citizens across the country are engaged in their communities through local arts and heritage; and
• Does the BCAH have any unexpected results?

Key findings

• There was no systematic collection of data and reporting on BCAH results beyond those gathered at the immediate and intermediate outcome level at the conclusion of funded projects, making it difficult to discern the achievement of the long-term, ultimate outcome. However, anecdotal evidence suggested that the projects funded through BCAH contributed to the engagement of citizens in their communities; created social benefits for communities; contributed to a greater appreciation of arts and heritage; led to the creation of longer-term partnerships and relationships; and provided longer-term benefits for artists.
• Unexpected positive results at the project level related to success beyond expectations in achieving the BCAH outcomes. Unexpected negative project results reported by recipients related either to the logistics (e.g., parking, congestion, noise), budgetary challenges; or volunteer burnout.

Projects have contributed to some extent to BCAH ’s ultimate outcome through sustained engagement of citizens across Canada.

The collection of data and reporting on the program’s ultimate outcome is absent for the three components of the program making it particularly difficult for the evaluation to assess long term results beyond some general observations. Recipients indicated that they do not track engagement beyond the event. However, anecdotal evidence suggests that to some degree BCAH projects are contributing toward the program’s ultimate outcome.

The 2012-2013 Arts & Heritage Access and Availability Survey noted that Canadians engage in arts and cultural events, not only through attendance but also by being personally involved.^{Footnote 43} Administrative data indicated that between 2011-2012 and 2014-2015, BCAH funded 3336 projects and events which collectively contributed to the involvement of citizens across the country. In summary, the wide reach of BCAH is illustrated by the fact that between 2011-2012 and 2014-2015 the program recorded a total of 38,999 partners^{Footnote 44}, 480,286 artists, artisans and heritage performers, 500,105 volunteers, 14,190,367 volunteer hours and 85,714,079 visits. In addition, projects and events occurred across the country and in communities of all sizes. In any given year, between 526 and 619 distinct communities across Canada were reached.

BCAH contributed to the levels of volunteerism in Canada. Statistics Canada reported that in 2013, 12.7 million Canadians or 44% of people, aged 15 years and older, participated in some form of volunteer work and that they contributed a total of 1.96 billion volunteer hours.^{Footnote 45} Through its projects and events, BCAH contributed to the 2013-2014 statistic, by engaging 140,896 volunteers for a total of 3,933,960 volunteer hours in that year.

Based on findings from the 2012 Arts & Heritage Access and Availability Survey on arts attendance, 83% of respondents indicated that they attended at least one type of live performance or arts event in the past year. The most popular (and most frequently attended) arts activities were “live art performances (63%), craft shows or fairs (55%) and arts or cultural festivals (52%)”. Three-quarters of respondents visited a heritage institution in the past year, with the most popular being historic buildings or sites (55%), museums or science centres (51%) and zoos, aquariums or botanical gardens (47%).^{Footnote 46} In 2012-2013, BCAH Local Festivals and Community Anniversaries projects contributed a total of 12 million visits to arts and heritage activities associated with their events.

In the absence of program data on the ultimate outcome, evidence of a sustained impact of BCAH on citizen engagement in communities across Canada is largely based on anecdotal evidence from key informant interviews and survey results. The evidence suggests that BCAH projects and events contributed to the ultimate outcome in the following ways:

• Created social benefits for communities. There was strong evidence from key informant interviews and surveys that BCAH projects had a direct impact on community identity and vitality and contributed to social cohesion. A significant number of recipients volunteered in their survey responses that their event had brought the community together, generated a sense of community, belonging, civic engagement and community pride; and/or that it had generated inter-cultural awareness.
• Contributed to an appreciation of arts and culture and heritage. The case study of a large festival event indicated it had an enormous impact in terms of broadening knowledge of percussion and encouraging learning. Since the operation of the festival, the number of percussion schools in the community leapt from one to 15 – 20. Many recipients also commented in interviews and through the survey that their event had an educational component or promoted awareness of local history and heritage.
• Led to the development of longer-term relationships, partnerships and social connections. Evidence from the case studies indicated that some projects have led to longer-term engagement and created benefits that extend well beyond the event itself. Both of the large case study festivals reported the creation of artist alliances and the emergence of new cultural businesses as a result of the success of their festivals. This evidence was supported by survey results. Some recipients reported that an unexpected outcome of their event or project was the development of longer-term relationships, partnerships and social connections. A few Legacy Fund recipients indicated that their project led to stronger working relationships with tourism organizations. A Legacy Fund recipient described how the community had come together to support the project with donations of artifacts, photos and paintings to the organization’s archives.
• Created longer term benefits for artists, artisans and heritage performers. Recipients provided examples of how artists, artisans and heritage performers have experienced longer-term benefits through participation in BCAH -funded projects, including local artists continuing to work together on subsequent projects – such as theatre or film productions and art shows. Some local film makers who premiered their film at the BCAH -funded festival have gone on to national and international success.

Most PCH officials and some recipients noted that the majority of festivals are recurrent, being able to secure partners year after year and the continued success of the event provided evidence of sustained community engagement. PCH officials also highlighted that the eligibility requirement that legacy projects must “be tangible and lasting with a useful life of at least ten years”^{Footnote 47} as contributing to the achievement of the ultimate outcome. Some recipients reported that as a result of the success of their event, there was an increased demand from their communities for similar events.

Unexpected results

Recipients reported unexpected results from the perspective of their projects. The most frequent positive unintended project impacts reported by BCAH recipients through key informant interviews and survey results included: benefits to artists, levels of engagement of volunteers and community partners beyond expectations; increased visibility/profile and interest in the organization and the community; increased civic engagement and pride; relationships, partnerships and social connections that have been sustained beyond the event; the generation of inter-cultural awareness; the educational impact of the event by increasing awareness and appreciation for arts, culture and heritage; greater media attention than anticipated; greater involvement of youth and non-traditional audiences (seniors, ethno-cultural groups) and the growth of the festival.

Unexpected negative project impacts identified by recipients in the surveys and interviews, were also at the project level and related either to the logistics (parking, congestion, noise), budgetary challenges; or volunteer burnout.

Core issue 5: demonstration of efficiency and economy

Under the Treasury Board Policy on Evaluation (2009) efficiency is defined as the relationship between the resources used and the outputs produced. The following sections summarize evaluation findings related to the efficiency and economy of the program.

Evaluation questions

• Are the resources dedicated to this program being used efficiently and economically to maximize the achievement of outcomes?
• Are there more efficient approaches to achieve BCAH outcomes?
• Is the BCAH program complementing or duplicating existing programs/initiatives?

Key findings

• The proportion of administrative costs to total expenditures for the program, for the period covered by the evaluation, was high at 18.7%. The proportion of administration costs to total expenditures for the administration of the Legacy Fund and program-related activities was 20.7%. It should be noted that the administrative costs for the Legacy Fund, administered by National Headquarters included part of the costs associated with the Director General’s office and the Strategic Policy and Research unit, as well as the costs associated with the program coordination function for the two regionally delivered components.
• In 2014-2015 the program lapsed $2.7 million of its G&Cs because Legacy Fund recipients were unable to spend their funds and a number of projects were not approved. Due to timing, the program was unable to reallocate funds. This accounted for the higher administration costs relative to total expenditures (25.1%) for that year.
• The proportion of administrative costs to total program expenditures to deliver the Community Anniversaries and Local Festivals components and related program activities averaged 18.2% for the four years covered by the evaluation. Administrative costs remained relatively constant ranging between 17.4% and 18.6%.
• Administration costs were higher than for similar PCH programs recently evaluated, including CAPF at 11% and CCSF at 7%. Comparing the volume of applications annually, the Local Festivals component of BCAH had a higher volume (1.4x) of applications than CAPF whereas CCSF has a higher volume of applications (4x) than the Legacy Fund component.
• BCAH had slight variances between budgeted and actual O&M expenditures except in 2012-2013, when BCAH had a deficit of 5.9%.
• Compared with the Community Anniversaries and Local Festivals, the Legacy Fund had higher administrative costs per application reviewed and projects funded reflecting differences in the characteristics and complexity of the projects funded, the level of demand and funding and the structure of the application review process.
• The Legacy Fund had the highest cost per outcomes reflecting the lower levels of citizen engagement for Legacy Fund projects.
• There are opportunities for greater efficiency, particularly in the processing of low risk, recurrent Local Festival files; consideration of multi-year funding; clarifying and simplifying application guidelines and processes; and reducing the lapse of Gs&Cs.
• Although there are other sources of financial support for festivals, anniversaries and legacy projects, the program complements other federal sources of funding and funding from other levels of government.

Since the previous evaluation, BCAH has implemented a variety of strategies to improve its efficiency; however, the proportion of administrative costs to total expenditures remain high at 18.7%. The proportion of administrative costs to total expenditures is higher than the previous evaluation (16.4%) and is higher than other similar PCH programs (7% to 11%).

Program documentation and interviews with PCH officials indicated that the program has applied a number of strategies in an effort to operate more efficiently. During the period covered by the evaluation, the program reviewed its operational processes, assessment tools and timelines and simplified budget templates, streamlined general application forms and guidelines, revised assessment tools and undertook a time and motion study to provide clarity on the process and timelines for the treatment of files to support consistency across the country. To better distribute the workload and budget, in January 2013, the program added a third application intake deadline for the Local Arts Festivals component.

BCAH has also participated in the corporate initiatives such as the Grants and Contributions Modernization Initiative (GCMI) directed at increasing efficiency by simplifying, standardizing and streamlining Gs&Cs business processes using a risk-based approach. Among the initiatives in which the BCAH program has participated are the triage process^{Footnote 48} for the treatment of files. The program’s target, in line with the GCMI target, was to achieve efficiencies by treating 40% of its applications as Basic. These applications would be subject to a streamlined process including limited assessment. However, the regions have encountered challenges in meeting this target. Administrative data indicates BCAH almost met the target (37.8%) in 2013-2014. However, in 2014-2015 only 27% of files were deemed to be Basic (Appendix D: Table 14 ). The program is working with the Centre for Excellence for Grants and Contributions to further refine the questions that determine the level of risk and the level of treatment required.

Administration costs remained high in spite of efforts to operate more efficiently. The administrative costs incurred by PCH for the management and delivery of the three BCAH components totaled $15,752,828 for the period 2011-2012 to 2014-2015 which represents 18.7% of total BCAH expenditures during this period (Appendix D:Table 19).^{Footnote 49} BCAH processes a high volume of applications. The majority of application received and approved annually are for the Local Festivals component which, for the period 2011-2012 to 2014-2015, averaged 991 applications per year with on average 752 projects funded per year.

BCAH administrative costs were higher than reported in the December 2011 evaluation (16.4%). However, this can be explained, in part, by the addition of the Legacy Fund component in 2009-2010, which involved more complex files.

National Headquarters administration costs for BCAH include the O&M and salary costs associated with the administration of the Legacy Fund component and the costs associated with program management and delivery activities undertaken at the national level such as policy development and implementation, planning, oversight and reporting. National Headquarters administration costs, therefore, included part of the costs associated with the Director General’s office and the Strategic Policy and Research unit, as well as part of the costs associated with program support functions. It excludes the salary and O&M costs associated with National Headquarters operational role in the delivery of the Local Festivals and Community Anniversaries components.^{Footnote 50} Average administration costs for the four year evaluation period was 20.7%. This figure fluctuated from a low of 16.7% in 2013-2014 to a high of 25.1% in 2014-2015 The decrease in 2013-2014 reflected a decrease in O&M expenditures relative to previous years. During 2014-2015 the proportion of administrative costs increased because of a lapse of $2.7 million in Gs&Cs while the O&M expenditure remained relatively constant.

The proportion of administrative costs to total program expenditures to administer the Community Anniversaries and Festivals components, delivered by the regions, include the O&M and salary associated with regional delivery as well as an estimate of the salary and O&M associated with National Headquarters support of the delivery of these two components. Administration costs averaged 18.2% for the four years covered by the evaluation and were relatively constant at between 17.4% and 18.6% over the four (Appendix D: Table 15).

The 2011 evaluation of BCAH compared BCAH ’s operational costs to that of three other PCH programs: Canada Arts Presentation Fund (CAPF), Canada Cultural Spaces Fund (CCSF) and Canada Cultural Investment Fund (CCIF). At the time, BCAH ’s operational costs compared favorably to these other programs. A similar analysis for this evaluation (Appendix D: Table 16) shows that the administration costs of the other programs have decreased since the 2011 evaluation. BCAH ’s administration costs at 18.7% are now significantly higher than those reported in the 2014 evaluation of these programs (7% to 11%).^{Footnote 51}

Comparing the Local Festivals component of BCAH with CAPF , which provides financial assistance to organizations that professionally present arts festivals or performing arts series (arts presenters) and organizations that offer support to arts presenters and, based on the results of the 2014 Grouped Arts Evaluation, BCAH has 1.4 times the volume of applications than CAPF . It averaged 991 applications annually and approved 863 applications. CAPF received on average 714 applications annually and approved 614. CCSF , which supports the improvement, renovation and construction of arts and heritage facilities, as well as the acquisition of specialized equipment, had four times the volume of applications compared with BCAH . It processed on average 177 applications annually and approved 108, compared with the Legacy Fund which averaged 42 applications annually and approved, on average 21 annually.

Administrative costs per application reviewed and project funded reflected differences in the characteristics and complexity of the projects funded, the level of demand and the structure of the application review process.

Overall, BCAH ’s administrative costs per application averaged $3,438 over the four years covered by the evaluation. Administrative costs per funded project averaged $4,676. For the Local Festivals and Community Anniversaries components, management and administrative costs averaged about $2,672 per application received and $3,536 per project funded over the four years (Appendix D: Table 17).^{Footnote 52}

For the Legacy Fund, the management and administrative costs averaged about $23,692 per application received and $52,059 per project funded over the four years. Factors that influenced the average cost are the level of demand relative to the funding available and the type, size and complexity of projects. The Legacy Fund projects has a cost per funded project which is significantly higher than the cost per application because the number of applications and the approval rate over the period of the evaluation were low relative to the Local Festivals and Anniversaries components. Between 2011-2012 and 2013-2014^{Footnote 53}, approximately half of all Legacy Fund applications received were approved and 70% of eligible applications. Furthermore, the type and size of projects also influences the costs per application and costs per project approved. Projects funded through the Legacy Fund are more complex, carry a higher level of risk and require more information, time and resources to administer.

Costs per outcome reflect the levels of citizen engagement for each component relative to the total amount paid. The higher levels of citizen engagement for Local Festivals projects offset the higher total amount paid, resulting in lower costs per outcome.

The cost per outcome is calculated by taking the total amount paid to recipients relative to each of the program’s intermediate outcomes (total number of volunteers, total number of artists, artisans and heritage performers and total number of participants). Costs per outcome were the lowest for Local Festivals and highest for Legacy Fund projects, reflecting the different levels of citizen engagement between the components. Legacy Fund projects engage fewer partners, volunteers and artists, artisans and heritage performers (Appendix D: Table 18). Legacy Fund projects are of longer duration, require fewer volunteers (but a greater commitment) and, because their unveiling event is of one day duration, they require fewer artists. As Legacy Fund projects are so different from the Local Festivals and Community Anniversaries projects, there may be additional performance measures that would better reflect legacy project outcomes.

In summary:

• Cost per volunteer for Local Festivals projects was $97, for Anniversaries $233 and Legacy Fund $3,836.
• Cost per artist, artisan and heritage performer for Local Festivals projects was $99, for Anniversaries $314 and Legacy Fund $3,088.
• Cost per participant for Local Festivals projects was $0.53, for Anniversaries $1.80 and Legacy Fund $296.80.

Actual G&C expenditures fluctuated, with the program lapsing approximately $2.3 million overall over the four year period of the evaluation. Most of this lapse occurred in fiscal year 2014-2015 and was due to the inability of Legacy Fund recipients to spend all their funds and, due to timing, the program being unable to reallocate the funds.

An analysis of the relationship between planned and actual Gs&Cs and O&M expenditures found that over the period under evaluation, reference levels for Gs&Cs were stable at approximately $17.7 million for a four year total of approximately $70.6 million. Actual Gs&Cs expenditures showed fluctuations, with the program lapsing approximately $2.3 million overall over the four year period of the evaluation. Most of this lapse occurred in fiscal year 2014-2015 when the program lapsed $2.7 million of its planned Gs&Cs budget. Program staff attributed this lapse to recipients’ inability to spend all their funds and to the fact that a number of projects were not approved. Due to timing, the program was unable to reallocate funds (Appendix D: Table 19).

Reference levels for O&M and salary amounted to approximately $15.6 million. With the exception of 2012-2013 which had an O&M deficit, annual variances between actual and planned O&M and salary expenditures were small, generally+2%, with overall deficit over the four years covered by the evaluation of 1% (Appendix D: Table 19).

Based on FTE data reported in the DPR s, between 2012-2013 and 2014-2015, BCAH actual FTEs remained stable. According to the DPR , the difference between planned and actual FTEs for 2014-2015 was due to changes in allocations to ensure alignment with the Treasury Board Secretariat definition of internal services and that resources are linked to each sub-program. (Appendix D: Table 20 ).

There are opportunities to improve efficiency

Funding recipients, PCH officials and evidence from the document and administrative document review identified the following opportunities to achieve greater efficiency:

• Streamline the treatment of recurrent and low-risk Local Festivals files. A streamlined processing of low risk, recurrent festivals files and the approval of more than one year of funding and further efforts to increase the number of Basic files, which are subject to a more streamline treatment, would lead to greater efficiencies.
• Reduce the length of time to notify recipients of the funding decision. Recipients consider the 26 week timeline for notification of the funding decision to be inexplicably long. Delays can seriously impact organizations’ ability to plan their event. Opportunities to streamline processes to reduce the time between application deadline date and notification of the funding decision could also lead to greater efficiency.
• Further simplify application processes and forms. Recipients generally, and Legacy Fund recipients in particular, expressed frustration with the complexity of the application process. Program staff indicated that there was a lot of back and forth interaction with Legacy recipients on their applications. Simplifying and clarifying the applications and process would contribute to a reduction in the number of incomplete files and the associated number of interactions between staff and recipients. The majority of PCH officials identified opportunities to further clarify the guidelines; for example, clarifying what is considered “local”; guidance on how to calculate volunteer hours; and reexamining the definition of “local artist”.
• Decrease the likelihood of lapsed Gs&Cs. The program lapsed Gs&Cs in 2014-2015 because Legacy Fund recipients were unable to spend their funds and because of timing, the program was unable to reallocate the funding. Although the lapse occurred in only one year, a mechanism to identify potential lapses earlier in the year would reduce the likelihood of lapses in the future and permit the reallocation of funding.

The program also anticipates achieving efficiencies through the EOS, launched in April 2016 for the BCAH program, which will enable applicants to apply on-line, creating efficiencies for PCH staff and recipients and through the delegation of authority to Directors General for the approval of Gs&Cs files under $75,000, which is expected to shorten the approval process.

The BCAH program complements rather than duplicates existing programs.

The 2011 evaluation of PCH’s Building Communities through Arts and Heritage Program ( BCAH ) recommended that the current program design be reviewed to address the perception of potential overlap between Local Festival and Legacy Fund components of BCAH and CAPF and CCSF respectively.^{Footnote 54} Follow-up to the recommendations concluded that although various PCH programs may share clients or provide funding for similar expenses, they have different objectives or are specifically designed to fill the niche in the need for support. For example, CAPF is focused on access to a variety of professional artistic experiences in their communities; while BCAH is focused on citizen engagement in their local communities. To ensure that no duplication occurs in practice, PCH allows an organization to receive funding under only one of the two programs for the same activity. Similarly, CCSF is larger in scope and focuses more on cultural organizations, while the BCAH Legacy Fund narrowly provides funding for capital projects in communities commemorating their centennial year and subsequent anniversaries in 25 year increments. There was minimal evidence from the survey data that BCAH projects secured funding from other PCH programs.

All three levels of government in Canada are involved, to varying degrees, in providing support to arts and culture and heritage. The evaluation found that BCAH projects accessed other federal funding, as well as P/T, non-profit and private sector funding. While a comprehensive environmental scan of other programs was beyond the scope of this evaluation, the evaluation evidence indicated that, BCAH ’s scope does not significantly overlap with those of other federal or P/T funding sources. The funding provided by BCAH compared with other funders can be relatively small, particularly for the Local Festivals component, and the objectives, focus and delivery structure of BCAH is distinct from those of other federal departments and P/T delivered programs. The objectives of these other programs can also be much narrower in scope or far more general. BCAH is also designed to encourage leveraging of PCH funding with funding from other sources, which reduces the potential for overlap or duplication.

Survey respondents reported accessing funding from regional development agencies of the federal government and from other federal departments and agencies including the Canada Council for the Arts, Service Canada, Canada Summer Jobs, CIC and Parks Canada. However, these funding programs have objectives that are distinct from that of BCAH .

BCAH -funded projects accessed funding from the Canada Council for the same project. Many of the Canada Council grant programs are targeted at individual artists and most commonly support the creation/production of artistic works, professional development, and residency and travel expenses. Other grants programs support art collectives, companies and other Canadian, non-profit arts organizations with funding for operating expenses, organizational development assistance, project assistance and presentation and dissemination assistance. BCAH’s funding, however, is targeted to local artists, artisans and heritage performers and volunteers.

The potential for overlap with Parks Canada’s National Cost-Sharing Program for Heritage Places (NCSPHP) was identified for the Legacy Fund. However, unlike NCSPHP, BCAH does not focus exclusively on national historic sites. A 2012 evaluation of NCSPHP by Parks Canada noted that demand has exceeded resources in each round of funding, with no more than 60% of eligible recipients receiving funding in any given year.

Evidence from the surveys found that many of the BCAH -funded festivals had an ethno-cultural or multicultural dimension and some Local Festivals projects also secured funding from CIC . However, as the objectives of BCAH and that of the Multicultural program were different, funding was determined to be complementary.^{Footnote 55} With the transfer of the Multiculturalism program to PCH from CIC in 2015, however, the department will need to undertake a strategic policy review to examine interdependencies across PCH policies and programs, including the BCAH program and consider how the each program can best contribute to PCH strategic outcomes and priorities and to the Government's broader diversity and inclusion agenda while ensuring that duplication does not occur between the two programs.

P/T governments provide funding for the development and promotion of arts and culture and heritage, through government departments or arm’s length agencies (e.g. provincial arts councils, foundations and boards). The range, type, objectives and structure of the programs, as well as their level of investment, vary significantly from region to region. Most programs have an economic and tourism focus. Therefore, they complement rather duplicate BCAH .

Other evaluation questions

Evaluation questions

Design and delivery: Were adequate management and administrative practices in place for effective delivery to meet the program’s expected results?

Performance measurement: Were the performance monitoring and measurement activities sufficient to support results reporting and evaluation? What, if any changes are required?

Official languages: Were all official languages requirements of the BCAH program met?

Key findings

• Recipients were generally satisfied with the design and delivery of BCAH ; however identified areas for improvement, including: providing more timely notification of the funding decision; improving the timing of the notification of funding in relation to the start date of the event; providing more stable funding; and further clarifying and simplifying the application guidelines and processes.
• The program’s funding model created challenges from the program’s perspective in terms of balancing funding between intakes. From the perspective of recurrent festival recipients, fluctuations in the levels of funding received from year to year created difficulties in terms of planning and securing funding from other partners.
• Through the Legacy Fund, BCAH funds projects involving historic sites designated nationally or under provincial legislation. However, aside from requiring applicants to confirm they have consulted and are in compliance with the Standards and Guidelines for the Conservation of Historic Places in Canada indicate, there was no evidence that recipients have to demonstrate compliance.
• The program collects and analyzes data on its immediate and intermediate outcomes, however does not collect data on its ultimate outcome.
• There are areas where data collection and analysis could be strengthened, including collecting data on the program’s ultimate outcome; social benefits to Canadians as a result of the program; projects which support GC diversity goals and PCH priority groups such as OLMC, youth, ethno-cultural or Indigenous communities).

Design and delivery

PCH key informants consider BCAH ’s delivery model, with regional delivery of the Local Festivals and Anniversaries component and national delivery of the Legacy Fund as appropriate, noting that regional delivery enhances effectiveness (e.g. being able to respond to regional needs) and delivery (able to work more directly with recipients).

Evidence from the document review, indicated that the Program has well established governance structure and practices in place that include: structures, processes, roles and responsibilities that are clearly defined, communicated and understood; national and regional review committees which provide assurance on the quality of assessment, risk management and due diligence in risk decision; and regular Manager’s Network meetings with regional and national staff to share information and discuss/resolve issues.

Overall, recipients were satisfied with the delivery of the BCAH program with the majority of survey respondents indicating that they were very satisfied or somewhat satisfied (89.5% to 95.2%). Except for the elements associated with the application (submission process and clarity of application guidelines), Legacy Fund recipients were the most satisfied (Appendix D: Figure 2). Similarly, recipients interviewed expressed their satisfaction with the program providing an overall rating of 4.5 out of five.

However, a number of key areas for improvement emerged from the surveys and key informant interviews, including: timeliness of notification of the funding decision (including the length of time it takes to review a file and make a decision and the timing of the decision relative to the event); application and reporting processes; fluctuations in funding; and the definition of “local”.

Timeliness of notification of funding decision^{Footnote 56}

Depending on the component, 25% to 40% of survey respondents expressed dissatisfaction with the timeliness of notification of the funding decision. Most of the comments were from Local Festivals recipients through the survey and interviews. Of Local Festivals respondents who provided comments, 38% (115/302) requested more timely notification of funding decisions. Their responses could be further subdivided on the basis of those who commented on the length of time it takes to process an application and notify the applicant of the funding decision (87/115) and the timing of the decision relative to the event date (28/115). The administrative data indicated that over the period covered by the evaluation, the program did not achieve full compliance with the 26 week service standard (Figure 1). Compared with 2011-2012 and 2012-2013, performance has declined for all three components in the last two fiscal years of the evaluation period. Recipients indicated that the 26 week timeframe is problematic in terms of planning, particularly for booking artists, artisans and heritage performers and in leveraging BCAH funding to secure other funding.

Figure 1: Service standard results – notification of the funding decision

Recipients often drew comparisons with the timelines of their other funders for the same project. Some recipients suggested that a 3-4 month turn-around-time for a decision on an application would be more acceptable.^{Footnote 57} In terms of comparable service standards for the Legacy Fund component, the Parks Canada’s National Historic Sites Cost-Cost-Sharing Program’s service standard is to issue official written notification of the funding decision within six months of the program’s application deadline date. If additional processing time is required they inform the applicant within that timeframe.^{Footnote 58}

Timing of the notification of decision

Local Festivals recipients also expressed dissatisfaction with the timing of the notification of the decision and the inconsistency in from timing year to year. Organizations encountered difficulties when the notification of funding was too close to the date that the event was scheduled to start.

The issue of timing was raised in the 2011 evaluation and since then the program has continued to collect data and monitor its performance. Evidence from the administrative data indicated that the program has shown notable signs of improvement toward reducing the assessment and approval period when compared with the previous evaluation. Some survey respondents also acknowledged that the delay in receiving notice has been shortened.

The 2011 evaluation noted that between 2008-2011, 44% of the festivals reviewed received their final approval less than 60 days before the event start date, while 32% of the projects reviewed received their final approval 90 days before the event start date. Over the period covered by this evaluation, 18% of the festivals reviewed received their final approval less than 60 days before the event start date, while 61% of the projects reviewed received their final approval at least 90 days before the event start date (Appendix D: Table 21).

Funding model

Survey results showed that 47% of Local Festivals recipients were dissatisfied with the funding they received relative to what was requested. Recipient interviews and the comments they provided to the survey indicated that they were also dissatisfied with the fluctuations in the levels of funding received year to year.

BCAH ’s funding model and funding formula determines individual contribution amounts each year. Documentation and interviews with PCH officials indicated that the funding model is intended to ensure fairness in the distribution of funding across the country and to enable the efficient processing of a large number of files.

Based on its funding model, BCAH funds all eligible applicants who have received a passing merit score. For the Local Festivals and Community Anniversaries component funding recommendations are determined by a funding formula which is applied to all files which are recommended for funding. The funding formula is a mathematical formula and is based on the annual Gs&Cs budget which is allocated to each specific intake for each fiscal year; P/T allocations (determined by 50% population and 50% demand); demand (analyzed eligible expenses) and merit ranking – the project score (based on factor such as number of partners, volunteer etc.). Merit rankings are facilitated by scoring tools/grids.

The current policy enables the program to support more applicants; however, depending on the above factors, can lead to fluctuations in the funding ratio (relationship between eligible demand and available funding) and to fluctuations in the amounts received by recipients year to year. Evaluation evidence from program documentation, survey results and comments from recipients indicated that a few were frustrated with the amount received relative to the amounts requested. This, combined with the inability to plan their events due to the fluctuations in the amounts received each year, led a few recipients to indicate that they were discouraged from reapplying.

PCH officials and program documentation identified a number of factors which can affect individual contribution amounts, and can generate fluctuations in amounts received each year, among them:

• the number of applications and eligible applications that achieve the minimum score;
• the total eligible amounts requested by applicants
• the number of “top-ups”: eligible requests less than $2,000 get 100% of request;
• the number of requests for the maximum amount of $200,000; and
• scoring in a number of areas, including: level of community engagement (partners and volunteers), arts and heritage content (opportunities for local artists, etc.), capacity of organization.

One of the tools to assist program staff to assess the merit of the applications are the scoring/population grids. Results feed into the funding formula. The program was unable to produce documentation on the underlying analysis and rationale associated with the population grids on which merit scores were based. The evaluation team observed that there may be some inherent biases favoring larger communities but without information or explanation of the rationale associated with the scoring grids, the evaluation cannot conclude definitively that this is the case. The program should consider reexamining the underlying logic and impact of the scoring grids to ensure that they do not penalize any one group.

Input from recipients through interviews and the surveys suggested they may not fully understand BCAH ’s funding model. The general perception among some recipients surveyed and those interviewed appeared to be that BCAH funding allocations are based on merit or need. Therefore, a few recipients expressed frustration with not receiving an explanation as to why they received less funding when they had demonstrated that they had enhanced their programming or provided a clear rationale for the amount of funding being requested. ^{Footnote 59} A few survey respondents shared that they asked for more in order to receive what they needed. A few requested more transparency on how they are evaluated and how funding is distributed. The majority of applicants interviewed who did not receive funding expressed a desire for more information on how they were assessed.

Evidence from program documentation and interviews with PCH officials indicated that over the period of the evaluation, the program has continuously made adjustments to equalize funding between intakes. This has included reconsideration in 2010 of the policy to fund all eligible applicants with a passing merit score with a recommendation that a 50% funding ratio be preserved by focusing support on the most meritorious events. However, at that time, the original policy to find all eligible applicants was reaffirmed.

PCH staff acknowledged some of the issues identified by recipients including the observation from over half of PCH officials (6/11) that an unintended impact of the model is that applicants who may be aware of how the model works will inflate their eligible requested amounts because they anticipate receiving less than requested, in effect, penalizing applicants who submit a realistic request but see their amount decreasing. Over a third of PCH officials (4/11) confirmed the fluctuations in the amount of funding allocated to repeat festivals year to year makes it difficult for recipients to plan. Over a quarter of PCH officials suggested that the funding formula is too restrictive in that once the formula is run, program officers cannot make adjustments to take into account factors such as the financial need of the organization. Over a quarter (3/11) suggested that the formula process takes a significant amount of time – from the point of submission of a recommendation, to the running of the formula to waiting for the briefing to be done. A few PCH staff noted that, as a result of the funding formula, it is difficult to explain to recipients the rationale for the amount of funding awarded.

Application and reporting requirements

Key informant interviews and survey findings suggest that BCAH ’s application and reporting requirements are onerous for some recipients, particularly when only small amounts of funding are provided. About 30% of Local Festivals and Community Anniversaries and about 40% of Legacy Fund recipients surveyed expressed dissatisfaction with the complexity of the application. A few recipients indicated the application as well as reporting requirements discouraged them from reapplying.

The majority of PCH officials interviewed indicated that although improvements have been made to the guidelines, there are opportunities for further improvements. Of the Local Festivals recipients who requested that the application process be simplified, half suggested that a “simpler” or “basic” application should be available to recurring festivals. A few also suggested that multi-year funding be available to recurrent clients.

Local Festivals survey respondents also commented on the application deadline, specifically the September deadline, suggesting that it be moved closer to the event because the deadline is too close to the end of the previous event. Recipients interviewed find it challenging to apply for the next year at the same time as they are preparing their final reports for the current year’s event.

Most PCH officials commented on the April intake, indicating that the application deadline is almost one year before the event takes place. Recipients are applying for the next event before the current event has occurred. PCH officials observed that this often results in discrepancy between planned activities and what is delivered. Staff encounter challenges in analyzing the application in the absence of a final report on the previous year’s event (since the event has not yet occurred) and having to rely on final reports from two years ago.

The level of reporting can also create challenges for volunteer-based organizations who may not have the skills or capacity for performance reporting. PCH officials noted that reporting requirements can also add to the administrative costs and can have a negative impact on the efficiency of the program. Therefore there is a need for a balance between ensuring sufficient information is available for accountability and evaluation purposes while not over-burdening recipients. The ongoing business process modernization being undertaken by the Centre of Expertise on Gs&Cs will be identifying barriers to streamlined service delivery and reporting requirements, particularly for grants, will be among the issues examined.

Legacy Fund: Standards and Guidelines for the Conservation of Historic Places in Canada (2003, 2010)

A review of projects funded through the Legacy Fund found that projects fell under the following broad categories, although some could fit under more than one category: Historic sites designated nationally or under provincial legislation (15); Century buildings of “local community significance” (16); Newer cultural infrastructure (including exterior spaces) to encourage arts and heritage activities (15); Statues and murals (19); and Municipal parks (8).

The evaluation identified 15 projects funded by BCAH involving sites that were designated nationally or under provincial legislation as an historic site. Program guidelines require that if a project is located in an historic place that has been formally recognized by a federal, provincial or municipal authority and/or is listed on the Canadian Register of Historic Places confirmation that the applicant has consulted and is in compliance with the Standards and Guidelines for the Conservation of Historic Places in Canada is required.^{Footnote 60}

Apart from the requirement to consult the Standards and Guidelines for the Conservation of Historic Places in Canada, there was no evidence of follow-up to ensure compliance with the standards and guidelines.

BCAH requires applicants to indicate that they have consulted the standards and guidelines and attest to being in compliance but there was no evidence of follow-up to ascertain compliance. Recipients provide an independent written feasibility study for major renovation/expansion projects with total expenses over $200,000 to determine the likelihood of success; however it was unclear if the feasibility study would confirm compliance with the standards and guidelines.

Consultations with Parks Canada, which operates the National Historic Sites Cost-Sharing Program which funds projects similar to those that were funded by BCAH ’s Legacy Fund, indicated that applicants must include as part of their application a narrative statement on how the proposed project will demonstrate adherence to the standards and guidelines. Compliance is also a consideration during the assessment of the application. Further, a pre-certification report signed by an architect with heritage experience (Parks Canada staff) is required prior to approval and following a site visit. A certification report is also required following a second site visit by the same professional, to ensure the completed work complies with the standards and guidelines.

Recipients of nine of the 15 projects, which were designated as designated nationally or under provincial legislation as an historic site, were non-profits; the remaining six were municipalities or provincial bodies. A few key informants noted during interviews that small, volunteer-run, non-profit organizations may not have the necessary expertise or capacity to manage an infrastructure project, or know what the priorities are for a renovation or restoration. The program should consider a mechanism, similar to that of Parks Canada, that provides assurance that the appropriate expertise has been consulted early in the project to assess compliance with the standards and guidelines and, once work is completed, that the integrity of the historic site was maintained. This could be achieved by requiring recipients to provide an engineering report from a qualified professional, with knowledge of the requirements of the standards and guidelines, before work commences and again once the project has been completed. Reports could be considered an eligible expense.

Performance Measurement

BCAH finalized its most recent PMERS in 2013. The program’s PMERS includes a program profile, logic model, performance measurement strategy, evaluation strategy and a section on risk assessment and management. There was evidence through the document review and interviews with PCH officials that the data is being used to assess trends and to make program adjustments. Performance against targets are reported in the DPR . The program also analyzes its data and produces an annual “Performance Story” report.

There are opportunities for the program to improve performance measurement to provide a more robust results story about BCAH ’s benefits to Canadians that aligns closely with the mandate of the Citizen, Heritage and Regions Sector.

BCAH ’s intermediate outcomes focus on levels of participation (number of partners, number of volunteers, number of local artists, artisans and heritage performers, number of participants). However, the evaluation found that PCH officials and recipients often referred to longer-term social benefits when describing the impact of BCAH projects. Evidence of longer-term social benefits (e.g., social cohesion, pride, attachment) is currently not collected by the program. However, collecting data and reporting on these benefits would enable the program to provide a more robust results story about BCAH ’s benefits to communities and Canadians.

Data on the immediate and intermediate outcomes is collected from final reports and compiled in a spreadsheet. Although data on outcomes collected from final reports are sometimes not provided, are incomplete or received late, program staff estimate that they receive 80% of the requested data.^{Footnote 61} However, recipients are not required to report on results beyond the fiscal year when their funding ends. Therefore, program data was not available on the ultimate outcome, falling outside of the timeframe under report. However, it may be feasible to collect information on the achievement of the ultimate outcome from recurrent recipients.

Program data is also managed using multiple databases. A merging of the two database would make data analysis and reporting less complex and labor intensive.

The recipient interviews and surveys indicate that the other organizations that provide funding to BCAH -funded projects also require that financial and other information. A scan of the applications and reporting requirements of these organizations indicate that they often ask for more data and information than does PCH. Therefore, there may be opportunities to request that organizations share financial and performance data already being collected by recipients in fulfillment of reporting requirements of other organizations.^{Footnote 62}

There is some evidence that other Canadian funders are collecting performance data on social benefits. The British Columbia’s Community Gaming Program includes the requirement to report on how the broader community benefited from the programs/services supported by community gaming grants. The Ontario Trillium Foundation provides surveys to its recipients to collect outcome data, some of which relate to social benefits.

The interview with the international expert noted that European cities have started collecting data on social benefits. For example, the Edinburgh Festivals Impact Study, indicated that increasing residents’ pride in their local area and supporting cultural diversity are two accepted social outcomes for festivals in European cities.^{Footnote 63} Collection of data on social impacts, may require that BCAH provide tools and advice to assist recipients to survey audiences, although there are good examples of indicators available for the two social impacts discussed. The Edinburgh study noted that volunteers are also a valuable population whose opinion should be counted. Monitoring and data capture should be an eligible expense. The program would have to plan how it will capture and analyze this data.

A scan of reporting requirements for organizations cited by BCAH Local Festivals recipients as funding sources indicated that to varying degrees other organizations require their recipients to report on economic benefits, financial data, levels of participation and visitor profiles. Although BCAH recipients are required to report on cash and in-kind support from municipalities and their community partners, they are not required to report on other sources of funding. Thus, it is difficult to ascertain how much of an impact BCAH funding has had on recipients and to disentangle this from other sources of funding. However, a scan of application guidelines for other funders accessed by BCAH projects indicates that they require projects to submit financial statements. The program should consider asking recipients to including their financial statements as part of their final reports.

Official languages

The program does not collect data on BCAH ’s compliance with official language requirements. However, all documentation about the program was distributed publically (application guidelines, forms etc.) in both official languages. Many of the organizations that were part of the case studies provided their on-line material in both official languages. Survey respondents were asked about their level of satisfaction with the availability of service in the official language of their choice. The large majority of responses for the anniversary (90.2%), festival (91.6%) and legacy (95.3%) component surveys reported being “very satisfied” or “satisfied”, with 80.3%, 81.9% and 90.5%, respectively, indicating that they were very satisfied.

Conclusions

Relevance

There is a continued need for federal support to organizations to engage citizens in their communities through arts and heritage activities. The conditions and challenges that led to the creation of BCAH persist today. The continued need for BCAH is underlined by the fact that there are few funding sources that support small local, specifically arts and heritage festivals and anniversary events which do not aim to increase tourism or promote professional artists. An aging arts and heritage infrastructure points to the need to continue to support the restoration, renovation or transformation of existing buildings or exterior spaces.

In the last two years covered by the evaluation, the number of applicants have declined for all components, however the decline has been more significant for the Community Anniversaries and Legacy Fund components. No clear reason emerged from the evaluation; however, a number of factors may be contributing to the decline, including the normal maturing of the program and fluctuations in historical events to celebrate. Alternatively, issues associated with program delivery such as the complexity of the application or small and fluctuating funding amounts may be discouraging some recipients from (re)applying.

The program is responsive as it funds all eligible Local Festivals and Community Anniversaries applicants that achieve the required minimum merit score. However, the demand for funding is greater than the level of funding available. Eligible applicants requested $152.5 million in funding and BCAH provided $63.3 million. The average amount of funding per project relative to the eligible amount requested for the four years covered by the evaluation was 41.3% for Local Festivals and 44.1% for Community Anniversaries projects.

To be more responsive to recipients, the program should address several issues related to its funding model, particularly for the Local Festivals component, including: the fluctuations in funding from year to year for recurrent festivals; the distribution of funding between large, well-established festivals and smaller festivals; and its performance with respect to its 26 week service standard to notify applicants of the funding decision.

The BCAH program objectives align with PCH strategic outcomes and with federal government priorities related to supporting and investing in vibrant communities and celebrating and supporting our arts and cultural communities. Ensuring that all Canadians have opportunities to engage in civic life, fostering pride and attachment and promoting Canada’s arts and heritage sectors continue to be an important responsibilities for the federal government. The program could be more responsive to GC and PCH priorities by targeting funding to GC or PCH priority groups or innovative projects.

Performance – achieving expected outcomes

The evaluation covered the period 2011-2012 to 2014-2015. During this period, BCAH projects contributed to the achievement of the expected outcomes of the program. During this period, PCH provided local organizations about 68.3 million to organizations in numerous communities across the country to carry out local festivals, community anniversaries and/or legacy projects contributing to the program’s immediate outcome – local organizations receive financial resources to carry out local festivals, community anniversary and/or legacy projects in their communities. BCAH funding played a significant role in helping organizations to leverage cash and in-kind support from local community partners, as well as non-local sources. Overall, for every $1 of BCAH funding, projects secured $5 of cash and in-kind support from municipal and community sources. However, funding fluctuations year-to-year for some recurrent festivals applicants impacted on their ability to plan their events and to leverage BCAH funding.

The program exceeded its targets for its intermediate outcomes. Projects supported through BCAH successfully established local partnerships, engaged local volunteers and local artists, artisans and heritage performers and offered events to the local community free or at low-cost, thereby contributing to BCAH ’s objective to engage citizens in their communities through arts and heritage, This contribution was less evident for the Legacy Fund projects. While more reliant on BCAH funding and on being able to leverage BCAH funding to secure other sources of cash and in-kind support, Legacy Fund recipients were less likely to consider BCAH funding as an effective mechanism to engage citizens in their communities. Legacy Fund projects also had lower levels of citizen engagement.

While it is difficult to ascertain the achievement of its ultimate outcome, as the program does not collect data on the extent to which citizens across the country are engaged in their communities, there is some anecdotal evidence of sustained citizen engagement in communities. Anecdotal evidence from surveys indicated that engagement in BCAH -funded arts and heritage festivals and events were found to benefit Canadians in many ways including improving their quality of life and contributing to social cohesion, pride and attachment to communities. However, these benefits are currently not being captured in the program’s logic model nor are recipients asked to report on them.

Performance – efficiency and economy

Over the period covered by the evaluation program staff implemented a variety of strategies to improve efficiency and the program participated in the corporate GCMI initiative; however administrative costs as a percentage of the total program budget has remained high, averaging 18.7% over the period covered by the evaluation. This is higher than reported in the last evaluation in 2011. Administrative costs as a percentage of total program budget, increased from 16.4% during the term of the previous evaluation (2007-08 to 2010-11) to 18.6% for 2011-2012 to 2014-2015. This is higher than that of similar programs at PCH, such as CCSF and CAPF at 7% and 11% respectively. BCAH ’s Local Festivals component had, 1.4 times the applications annually than CAPF . However, when the Legacy Fund was compared to CCSF , CCSF had four times the number of applications annually than the Legacy Fund. The proportion of administrative costs to total expenditures was slightly lower for the Local Festivals and Community Anniversaries components (18.2%) than for the Legacy Fund (20.7%)^{Footnote 64} component. However, the Legacy Fund is comprised of more complex files.

The administrative costs per application reviewed and project funded varied across the three components. Factors that impacted on efficiency included demand, the type and complexity of applications and the structure of the application review process. Administrative costs per funded project were highest for the Legacy Fund projects, reflecting their complexity, the lower number of applications and the lower approval rate of applications. Costs per outcome (number of volunteers, number of artists, artisans and heritage performers and number of participants) were also higher for the Legacy Fund projects, reflecting their higher funding levels but, due to their nature (short one day unveiling event, fewer volunteers but greater commitment), lower levels of citizen engagement.

In 2014-2015, the program lapsed $2.7 million in Gs&Cs, largely as a result of Legacy Fund recipients’ inability to spend all their funds; and a number of projects were not approved and, due to timing, funds could not be reallocated.

The program should seek further opportunities to improve efficiency and economy by increasing the number of files that receive basic treatment; introducing a more streamlined processing of recurrent festival applications of low dollar value and deemed to be low risk; improving the timelines for notification of the funding decision; clarifying the application and processes, to reduce the number of incomplete files and the number of interactions between applicants and program staff; and reducing the likelihood of lapsed Gs&Cs. Staff anticipate achieving additional efficiencies as the program transitions to the Enterprise Online System and benefits from the delegation of authority of files under $75,000 to Director general.

While there are some similarities between BCAH and other PCH ( CCSF and CAPF ), federal government and P/T programs, they complement each other by setting distinct objectives, having different priorities, funding different expenses and/or employing different delivery models. The potential for overlap between BCAH and CCSF and CAPF , identified in the 2011 evaluation, have been addressed and strategies have been taken to minimize the potential for overlap and duplication.

Performance – other evaluation issues

Overall, recipients were satisfied with the design and delivery of the program. However, there may be opportunities for the program to reduce the application processing times for all components and the timing of the funding decision in relation to the start date of recurrent festivals. Many recipients, in particular local Festivals recipients, expressed dissatisfaction with the service standard for notification of the funding decision and, in the case of recurrent festivals, the timing of notification of the decision in relation to their event date. The administrative data review confirmed that the application processing times exceeded the published 26 week service standard for funding decision timelines. Since the 2011 evaluation, the program has improved the timing of the funding decision in relation to the start date of the event, however based on continued dissatisfaction expressed by some recipients, there may be opportunities for further improvement.

The program should review its funding model and the assessment tools to ensure that they do not discriminate against festivals in small population centres or rural areas. As well, the program should seek to address the fluctuations year-to-year experienced by some recurrent festivals. Based on BCAH ’s funding model for festival and anniversary applications, all eligible applicants who achieve the minimum required merit score are funded. However, in the context of set resources and shifting demand (e.g., the number of eligible applicants and eligible requested amounts) this policy has led to fluctuations in the funding outcome for individual recipients from year to year. This makes it difficult for recipients to plan their event or to use BCAH funding to leverage other sources of funding.

The program should seek opportunities to further streamline and simplify application forms and processes. Some organizations experienced greater challenges in preparing applications for funding due to their small size or because they were volunteer-run. Some recipients indicated that the requirements of the application were too detailed in relation to the amount of funding being requested. The recurrent Local Festivals recipients requested that a simpler or basic application be available to those with an established funding history with the program and that the application deadline be moved closer to the event.

The program should consider mechanisms to ensure that funded legacy projects comply with the Standards and Guidelines for the Conservation of Historic Places in Canada for projects that involve the restoration, renovation or transformation of a building designated nationally or under provincial legislation as a historic site. Between 2011-2012 and 2014-2015, about 20% of projects funded through the Legacy Fund involved historic sites designated nationally or under provincial legislation. While program guidelines require applicants to confirm that they have consulted and are in compliance with the standards and guidelines, the program does not obtain documentation confirming that projects have complied with the standards and guidelines.

BCAH has a PMERS and performance measures and outcomes are documented, actively monitored and reported. Over the period covered by the evaluation, the program produced an annual performance report and there is evidence that the performance data is being used in management decision-making. However, there are opportunities for the program to improve its performance measurement by collecting data on its ultimate outcome, by providing a more robust results story on BCAH ’s benefits to Canadians and by obtaining additional performance information, in particular financial information to permit analysis of project funding sources. In the qualitative data collected through the surveys, recipients described the social benefits which resulted from their projects. While these are not measured by the program, they can provide a meaningful representation of the longer-term impact on Canadians of BCAH events and festivals. There is evidence that a few Canadian jurisdictions and some European cities ask their funders to describe the social benefits of their projects on participants.

BCAH projects secure funding from a variety of sources for the same project. The program collects data on cash and in-kind support from municipal and local partners, however this is an incomplete picture of the funding that projects receive. In the absence of a complete picture of project funding sources, it was difficult for the evaluation to ascertain the impact of BCAH funding relative to other sources of funding, and funding overlap with other levels of government and the not-for-profit sector. A scan of the reporting requirements of funding sources identified through the surveys indicated that, among other things, most other funders require projects to report financial data. Where data is already being collected by projects, the program should consider requesting this information as part of their reporting requirements.

The program does not collect information and/or report on festivals that support GC diversity objectives or PCH priority groups such as OLMC, youth and Indigenous or ethno-cultural groups. Qualitative survey data, indicated that many of the BCAH funded festivals have an ethno-cultural and language dimension.

Recommendations and management response

Relevance

Issue #1: continued need for program