Evaluation of the Capital Experience Program 2013-14 to 2017-18

Evaluation Services Directorate
January 6, 2022

On this page

List of tables

Alternate format

Evaluation of the Capital Experience Program 2013-14 to 2017-18 [PDF version - 1.3 KB]

List of acronyms and abbreviations

ADM
Assistant Deputy Minister
CCP
Celebration and Commemoration Program
CCPO
Capital Celebrations and Program Operations Directorate
CCR
Canada’s Capital Region
CHIN
Canadian Heritage Information Network
CIC
Capital Interpretation and Commemorations Directorate
CE
Capital Experience Program
CLAC
Christmas Lights Across Canada
DGO
Director General Office
DRF
Departmental Results Framework
ESD
Evaluation Services Directorate
FAA
Financial Administration Act
FTEs
Full Time Equivalents
Gs&Cs
Grants and Contributions
GBA Plus
Gender-Based Analysis Plus
GoC
Government of Canada
HR
Human Resources
KII
Key Informant Interview
LGBTQ2+
Lesbian, Gay, Bisexual, Transgender, Queer, Two-Spirited, Plus
MECCE
Major Events, Commemorations and Capital Experience Branch
MoU
Memorandum of Understanding
NCC
National Capital Commission
O&M
Operations and Maintenance
PAA
Program Alignment Architecture
PCH
Department of Canadian Heritage
PIPs
Performance Information Profiles
PPS
Parliamentary Protection Services
PSES
Public Service Employee Survey
PSPC
Public Services and Procurement Canada
SMEC
Sport, Major Events and Commemorations Sector

Executive summary

Program description

The Capital Experience (CE) Program supports the production of events and activities as well as the development and delivery of new monuments, public art, interpretation and information services in Canada’s Capital Region (CCR). CE supports such events as Winterlude, Canada Day celebrations, Christmas Lights Across Canada (CLAC), and the Sound and Light Show on Parliament Hill. The objective of the CE Program is to foster feelings of pride and belonging amongst Canadians towards their Capital, while increasing their overall awareness of Canada’s Capital Region as a destination where people can experience Canada’s heritage, culture and achievements.

Evaluation approach and methodology

This evaluation is the first for the program and it covers a 4.5-year period from 2013-14 to 2017-18. Based on input from program management gathered during scoping interviews and given that the program had been recently transferred from the National Capital Commission (NCC) to Canadian Heritage (PCH), the evaluation focused on questions related to organizational change.

The evaluation was guided by the Burke-Litwin Organizational Performance and Change Model. This model describes the linkages among the key factors that affect organizational performance and determine how change occurs within an organization. The core dimensions that were explored included: external environment; mission and strategy; organizational culture; structure; and systems. Methodologies used for the collection and analysis of data included document, administrative data, and literature reviews, key informant interviews (KII), and a media content analysis.

Findings

Program results

The evaluation team analyzed existing performance measurement data to provide an overview of the immediate and intermediate results over the period. Further, a media analysis was done to examine national perceptions of CE programming.

The immediate outcome of Canadians had access to events and activities in CCR that celebrate Canadian identity and reflect Canada’s diversity was met, as well as its intermediate outcome that Canadians participate in events and activities that celebrate Canadian identity and reflect Canada's diversity in the CCR. Attendance at events was generally stable over the evaluation period, with a high in 2017, due to the 150th celebrations.

The results of a 2015 satisfaction survey on Canada Day in the Capital indicated that a majority of respondents (71%) agreed that the activities were representative of Canada, and more than half of respondents (62%) agreed that their visit gave them a better appreciation of Canada's cultural and social diversity. In 2017, a poll conducted with funding from Canada 150 demonstrated that nearly two thirds of respondents (63%) agreed that the festivities were representative of Canada.

The CE programming also contributed to Canadians having access to information and orientation to help them experience the sites and symbols of CCR. The number of requests for information from visitor services increased every year between 2013-14 (515,214) and 2017-18 (865,982). The programming also met its expected intermediate result of Capital Interpretation and Commemorations (CIC) that Canadians experience CCR and its symbols and sites.

Over the period covered by the evaluation, the programming received numerous external awards. The program’s results were also demonstrated through media analysis and show that 75% of the media mentions of program activities were positive. There was an increase in media mentions over the period of the evaluation, from 262 articles in 2013-14 to 365 articles in 2017-18.

External environment

Various external challenges impacted the CE Program, including strengthened security on Parliament Hill, involvement in the Canada 150 celebrations, increased media attention to program events and projects, and more frequent adverse weather conditions. The CE Program implemented several strategies to address these factors and improve program delivery, including developing a strategy for public communications regarding weather conditions and providing media training to employees to manage the impact of increasing media attention to events and monuments. The program also increased its security protocols, while improving visitor experience, based on lessons learned from previous events.

Mission and strategy

The CE objectives are well aligned with PCH’s mandate and priorities, particularly with respect to diversity and inclusion and strengthening Canadians’ sense of connection to each other and to Canada. It supports the overall mandate of PCH’s Major Events, Commemorations and Capital Experience Branch (MECCE), to strengthen Canadians’ sense of connection to each other and to Canada, advance awareness of and appreciation for Canadians’ shared values, cultural diversity, symbols, and institutions, and provide opportunities for Canadians to engage in public commemorations and celebrations of Canadian events and accomplishments.

The program demonstrated support for cultural diversity and equity-deserving groupsFootnote 1 through its events and activities as well as through internal departmental policies and practices. For example, there was evidence of volunteer recruitment from a diversity of ethnocultural communities.

Although the mandate and the objectives of the transfer are clearly articulated in multiple official documents, some internal and external stakeholders raised questions related to the program’s mandate to promote the Capital.

Organizational culture

There is evidence that the integration of the CE Program within PCH and MECCE has been mostly effective and that early challenges have stabilized over time. There was successful continued delivery of events and projects throughout and after the transfer, which was seen by key informants as evidence of successful integration. There was evidence of plans to further address integration of the CE Program within PCH and specifically within MECCE.

Structure

The CE Program is well coordinated and has led to collaborations with other MECCE programs, particularly related to event delivery. The CE Program has many partnering arrangements and collaborations at the project level, both internal and external to PCH, which has facilitated program delivery. While the program’s design and delivery model offers features that support efficiency, the evaluation identified some opportunities for improvement including through the clarification of some roles and responsibilities, and by ensuring a common approach for strategic communications and collaborations.

Systems

The evaluation determined that one of the unintended impacts of the transfer of the CE Program to a federal department was the impact on the immediate efficiency of program delivery with the longer timeline required for federal contracting processes. Although major events were successfully delivered, the transfer may have led to more last-minute work in the short term. The transfer also affected the extent to which sponsorships could be used for program activities. The increased demands of the 150th celebrations negatively impacted sponsors’ commitment for subsequent years.

Recommendation

The evaluation recommends that the assistant deputy minister for Sport, Major Events and Commemorations Sector further develop, communicate and implement strategic priorities for the program, which will guide and define the program’s development and delivery.

1. Introduction

This report presents the findings from the evaluation of PCH’s CE Program. The evaluation was carried out as indicated in the PCH Evaluation Plan, 2018-19 to 2022-23. It was conducted in accordance with the Treasury Board Policy on Results (2016) and Directive on Results (2016).

This was the first evaluation of the program and covers a four-and-a-half-year period from September 30, 2013 to March 31, 2018. Given the transfer from the NCC to PCH in 2013 and with input from senior management during scoping interviews, the evaluation focused on key organizational and change management issues that might have affected program performance.

2. Program profile

Unlike the majority of PCH programs, the CE Program does not distribute funding through grants and contributions (Gs&Cs). Rather, the program organizes and delivers activities using operations and materials (O&M) funding in the following areas: major events and celebrations, commemorations, public art, interpretation and promotion of sites and symbols of national significance, representation of the provinces and territories in CCR, youth activities, and visitor services.

2.1. Program history

On September 30, 2013, as a result of Budget 2013 and subsequent amendments to the National Capital Act and the Department of Canadian Heritage Act, specific duties and functions of the NCC were transferred to PCH. According to a Memorandum of Understanding (MoU) between the NCC and PCH, the purpose of the transfer was to “promote Canada’s Capital Region through its Capital Experience” Program to ensure that a broad, national perspective be brought to celebrations and that events draw on the cultural and social fabric of the whole of Canada.

2.2. Program activities, objectives, expected outcomes, and results

2.2.1. Objectives and Expected Outcomes

The objective of the CE Program is to foster feelings of pride and belonging among Canadians. The CE Program objective and its expected outcomes are shown in Table 1. The Logic Model that links program activities to the expected outcomes can be found in Annex A.

Table 1: program objective and expected outcomes
Objective Immediate outcomes Intermediate outcomes
To foster feelings of pride and belonging among Canadians towards their Capital, while increasing their overall awareness of CCR as a destination where people can experience Canada’s heritage, culture, and achievements.
  • Canadians have access to events and activities that celebrate Canadian identity and reflect CCR.
  • Equipment and technical expertise are available for events in CCR.
  • Canadians have access to information and orientation to help them experience the sites and symbols of Canada’s National Capital.
  • CCR features interpretive content, monuments, commemorations, and public art that reflect Canadian identity, culture, and values.
  • Canadians participate in events and activities in CCR that celebrate Canadian identity and reflect Canada’s diversity.
  • Canadians experience CCR and its sites and symbols.

Following its transfer to PCH, the CE Program appeared in the Department’s Program Alignment Architecture (PAA) as a sub-program under the program “Attachment to Canada” and supported the Department’s second strategic outcome, “Canadians share, express, and appreciate their Canadian identity.” With the approval of the Departmental Results Framework (DRF) in 2017-18, the CE Program fell under Core Responsibility 2 – Heritage and Celebration and now supports the ultimate result “Canadians are engaged in celebrations and commemorations of national significance”.

2.2.2. Activities

The core program activities that the CE Program carries out are related to the development, promotion and production of events, monuments, public art and services in CCR. Major events include Winterlude, Canada Day celebrations, and Christmas Lights Across Canada. The program also supports public art, commemorations, monuments and visitor services in CCR. The program does not deliver grants and contributions.

During the period of this evaluation, and in preparation for the extensive Canada 150 celebrations, CE Program worked with and supported the functions of the Canada 150 Federal Secretariat and participated in a Special Project Team from 2015 to 2018.

Events and activities are currently measured via physical and virtual attendance numbers, and the number of media mentions. For major events such as the 150th celebrations, regional economic impacts are also examined as appropriate.

2.3. Program management and governance

Accountability for the CE Program is with the assistant deputy minister of PCH’s Sport, Major Events and Commemorations Sector. The director general of the MECCE Branch oversees program delivery, which is undertaken jointly by two directorates: Capital Celebrations and Program Operations (CCPO); and Capital Interpretation and Commemorations (CIC). Annex C presents more details on the organizational structure.

CCPO is responsible for:

CIC is responsible for managing:

The NCC is a key partner of the CE Program. A joint MoU signed by the deputy minister of PCH and the chief executive officer of the NCC in October 2013 governs the relationship between the two partners. The MoU was intended to ensure that the ongoing activities of the CE Program were maintained without interruption as of, and following, the date on which the CE Program mandate was transferred to PCH. In the MoU, both entities committed to ensuring the ongoing success of the CE Program in its commemoration of the milestones that have helped to shape Canada, as a country, as well as the implementation of a broad, national perspective in major events in CCR.

According to the MoU, PCH is the overall project manager for the CE Program, while the NCC collaborates, supports, advises, and provides required approvals in certain areas such as site approvals for events, and design and site selection for monuments. NCC also manages and maintains NCC lands and structures on these lands, including monuments and pre-existing permanent public art installations.

In 2019, the responsibility for the NCC was transferred to the Minister of Public Services and Procurement Canada.

2.4. Program resources

On September 30, 2013, assets and liabilities related to the CE Program were transferred from the NCC to PCH (including assets with an initial book value of $4,698,000). CE Program assets consisted mostly of production services, information technology, and communications equipment used for events and activities. In addition, CCPO provided overall management of inventory for festival equipment and hardware that could be rented through its Rent and Loan Program. Assets such as existing monuments, public artworks and interpretive structures were not transferred from the NCC.

As outlined in Table 2, over the period of this evaluation, actual spending on the CE program was approximately $77.6 million including salaries, goods and services, and O&M.Footnote 2 Lower expenditures in 2013-14 reflect that the CE Program was transferred to PCH mid-way through the fiscal year. PCH’s involvement in the 2017 Canada 150 events accounts for higher expenditures in that fiscal year and in the previous year leading to 2017. This funding was a one-time investment for Canada 150.

Table 2: program spending, 2013-14 to 2017-18 (actuals)Footnote 3
Resource 2013-14 2014-15 2015-16 2016-17 2017-18 Total ($)
Salary ($) 2,501,015 4,998,238 5,428,257 5,452,556 7,481,199 25,861,265
Goods & services ($) 3,283,233 10,491,927 9,027,291 11,085,549 16,017,157 49,905,157
O&M ($) 177,676 337,287 354,768 429,235 488,282 1,787,248
Total ($) 5,961,924 15,827,452 14,810,316 16,967,340 23,986,638 77,553,670

Source: PCH financial data

3. Approach and methodology

The evaluation was led by PCH’s Evaluation Services Directorate (ESD), with consultant support for the media content analysis and the development of the report. This section outlines the evaluation approach and methodology including scope, timelines, calibration, evaluation questions, data collection methods, limitations, and mitigation strategies.

3.1. Scope, timeline and quality control

The evaluation covered the period from September 30, 2013, to March 31, 2018. Based on input from program management as well as timelines and resources available, the evaluation focused on the CE Program mandate and operational challenges following its transfer from the NCC to PCH. While the effectiveness and efficiency of the programming was not an explicit focus of the evaluation, the examination of operational challenges led to addressing these issues.

The evaluation used an organizational assessment model, the Burke-Litwin Organizational Performance and Change Model, as a framework to guide the evaluation, including the development of evaluation questions, data collection, analysis of findings, and reporting. The Burke-Litwin model describes the linkages among the key factors that affect organizational performance and determine how change occurs within an organization.

While the Burke-Litwin model was generally relevant to the CE Program, the context of organizational change management in the public sector required some adaptation. The factors that were examined for this evaluation were external environment, mission and strategy, organization culture, structure, and systems. Two additional dimensions of the Burke-Litwin model, tasks and individual skills and employees’ needs and values, were initially included in the evaluation scope. However, due to the time gap between the beginning of the project and its completion, any initial challenges with the transfer had been resolved, and there were no substantial findings from these two dimensions of the model to include in the final report. Additional information on the Burke-Litwin Model can be found in Annex D.

The evaluation of the effectiveness of activities funded through the Canada 150 Fund was outside of the scope of the evaluation; however, it must be noted that the public interest, media attention, and government investment related to Canada 150 events were a significant context for both the 2017-18 fiscal year and the previous fiscal year and heavily influenced areas such as attendance, media views and funding levels.

3.2. Calibration

The evaluation was calibrated as follows:

3.3. Evaluation questions

Table 3 outlines the specific evaluation questions by Burke-Litwin dimension that were used to guide the evaluation, including the development of data collection instruments and the analyses. Program results were also examined and are to be found in section 4.6 of this report. More details related to the indicators and data collection methods can be found in the Evaluation Framework (Annex B).

Table 3: evaluation questions by Burke-Litwin dimension
Burke-Litwin dimension Evaluation questions
1. External Environment

1.1. What external environmental challenges and opportunities exist which affect the CE Program?

1.2. How has the CE Program responded to these challenges and opportunities?

2. Mission and strategy

2.1. Do program representatives and relevant stakeholders have a clear understanding of the objectives and mandate of the CE Program following its transfer from the NCC to PCH?

2.2. To what extent are the CE Program mission, mandate, objectives and priorities, as currently stated, aligned with: the PCH DRF, core responsibilities, strategic outcomes and PCH and Government of Canada priorities; Sport, Major Events, and Commemorations Sector mission, mandate, objectives and priorities; and Major Events, Commemorations, and Capital Experience Branch mission, mandate objectives and priorities?

2.3. Does the CE Program support equity-deserving groups?

3. Organizational culture 3.1. How effectively has the CE Program integrated with PCH and MECCE?
4. Structure

4.1. Is the current CE Program design and delivery model the most effective and efficient for the delivery of the Program’s mandate?

4.2. Are there alternative delivery models that could be considered?

5. Systems (policies and procedures, human resources and resource planning)

5.1. What are the unintended impacts on program delivery of the CE Program transfer to a federal department?

5.2. To what extent do program resources (such as human and financial) enable the effective delivery of the CE Program activities and the achievement of its outcomes?

3.4. Data collection methods

This evaluation used a mixed-method approach including a document review, a literature review, administrative data analysis, interviews with program representatives and key stakeholders, and a media content analysis. The following provides a description of each of the data collection methods.

3.4.1. Document review

The evaluation team reviewed a total of 38 Government of Canada documents and publications related to the delivery and mandate of the CE Program to collect background and contextual information needed to understand program operations. The review included existing program documentation, policies, MoUs, priority documents, corporate documents, industry reports, and relevant external documentation.

3.4.2. Administrative data review

The evaluation team undertook an analysis of the following CE Program administrative data:

3.4.3. Interviews with program representatives and key stakeholders

A total of 29 interviews were conducted as part of the evaluation and included individuals responsible for managing and delivering the program’s various components, as well as key CE Program partners. Respondents included six external stakeholders from other government departments, the NCC, and partner agencies and organizations; nine internal services representatives; eight CE Program management representatives; and six CE Program staff.

3.4.4. Literature review

The literature review entailed an analysis of published literature and websites, specifically examining models for delivery of similar programming among four capital cities: Washington, Canberra, Québec, and Regina. The four capitals were selected based on several factors, including availability of information, the similarity of the legal-administrative frameworks and the programs implemented, and inclusion of some Canadian provincial capitals to allow for comparisons within a Canadian context.

Additionally, the literature review included analysis of research related to organizational change management in the public sector, which helped to contextualize the impacts of the CE Program transfer from the NCC to PCH. The following summary of theories was used to identify factors that could not be expected to be addressed by PCH and the CE Program in their management of change as they were beyond their control:

  1. Public organizations are more sensitive to their environment
    • Permanent change to adapt to the needs of all service delivery clients
    • Greater vulnerability (for example, dissatisfaction, change in public policies, political directives)
  2. Public organizations have less control over change and its direction
    • Decision-making is more political than managerial
    • Imposition rather than explanation of the merits of changes implemented
    • Actions are limited by a strong regulatory framework
  3. There are a considerable number of parties with issues and interests within public organizations
    • Mutually legitimate expectations that can easily be in conflict

3.4.5. Media content analysis

An analysis of media content was undertaken to provide an understanding of key themes related to the CE Program, and included stories that were written, shared, and discussed in Canadian news media over the evaluation period. The media scan included English and French print and broadcast media sources, as well as other specialized content identified by the CE Program; an analysis, synthesis, and triangulation of the data collected; and the development of a draft and final technical report on the findings/results of the media analysis. Stories were sampled from April 1, 2013 to March 31, 2018, which included the period leading up to and following the transfer of the CE Program from the NCC to Canadian Heritage. In total, 1,467 articles in the quantitative analysis and 109 stories in the qualitative analyses were reviewed.

ProQuest was the main tool used for the media content analysis, supplemented by online searches for select media sources. Specialized journals covering monuments and art displays were also included in searches for relevant monuments and art installations and searches were also performed in le Droit to capture French language coverage from CCR. Finally, limited Google News searches were performed to supplement the other searches. The quantitative sample consisted of a total of 1,467 stories, including 1,288 English stories and 179 French stories. A qualitative sub-sample of 83 stories with substantive discussion on relevant topics was then drawn from this sample.

Further details regarding the media analysis strategy and findings can be found in Annex E.

3.5. Evaluation limits and mitigation

The following outlines the key constraints and limitations of the evaluation process.

Table 4: limits and mitigations
Limits Mitigation

Evaluation project delays.

Due to challenges related to the COVID-19 global pandemic, several years had passed between the period covered by the evaluation and the finalization of the report. Multiple adjustments and improvements have since been implemented in the CE Program.

Both the program and evaluation teams changed personnel, and there were limitations on the knowledge of the evaluation context and on corporate knowledge related to transfer.

The evaluation team and the program maintained communication over the period to ensure the ongoing relevance of the findings and conclusions.

Limited evaluation evidence to analyze the public art component of the CE Program.

This component was rarely mentioned in the KIIs, documents, or media analysis.

Public art is a relatively small component of the programming; the evaluation draws no specific conclusions related to public art.

4. Findings

This section presents a summary of CE Program results over the evaluation period followed by the analysis of the program using the Burke-Litwin Organizational Performance and Change Model.

4.1. Program results

While this evaluation was designed to focus on the transfer of the CE Program from the NCC to PCH, the evaluation team analyzed existing performance measurement data to provide an overview of the immediate and intermediate results over the period.

CCPO Results

CCPO achieved its immediate result that Canadians have access to events and activities in CCR that celebrate Canadian identity and reflect Canada’s diversity and its intermediate outcome that Canadians participate in events and activities that celebrate Canadian identity and reflect Canada's diversity in the CCR.

Tables 5 and 6 describe trends in social media impressions and broadcast viewership between 2014-15 and 2017-18 for Canada Day, Winterlude and Christmas Lights Across Canada. The increasing numbers over time indicate an overall increase in access to events and activities in CCR.

Table 5: CCPO Program Results, social media, 2014-15 to 2017-18
Social media impressions 2014-15 2015-16 2016-17 2017-18
Canada Day 3,423,677 691,858 1,054,758 5,667,407
Winterlude - 3,023,143 5,123,420 3,404,830
CLAC 379,949 - 2,148,511 1,308,378

Source: Program data

Table 6: CCPO Program Results, viewership, 2014-15 to 2017-18
Viewers of Broadcasts 2014-15 2015-16 2016-17 2017-18
Canada Day 2,601,334 - 1,771,240 15,000,000Table 6 note *
Winterlude - - 1,134,000 4,453,000
CLAC (Facebook Live) - - 200,000 160,000
Table 6 notes
Table 6 note *

Source: Program data higher than usual attendance number was largely due to the 150th celebrations.

Return to table 6 note * referrer

Annual, in person attendance estimates for these events was generally stable over 4.5 years, with the exception of higher numbers in 2017-2018, due to the 150th celebrations. Average attendances were:

The results of a 2015 satisfaction survey on Canada Day in the Capital indicated that a majority of respondents (71%) agreed that the activities were representative of Canada and more than half of respondents (62%) agreed that their visit gave them a better appreciation of Canada's cultural and social diversity. In 2017, a poll conducted with funding from Canada 150 demonstrated that nearly two thirds of respondents (63%) agreed that the festivities were representative of Canada.

Economic Impacts

There is evidence that the CE programming contributes to positive economic impacts for the CCR. A satisfaction survey of Canada Day 2017 in the Capital was carried out by a third party on behalf of PCH and financed with funds from Canada 150. The results of the survey underscored the economic impact of this event, particularly for tourism. According to the survey, Canada Day influenced the decision of visitors to come to CCR by 90% in July 2017, which was a significant increase from 69% in 2015. The majority of respondents (88%) stayed at least one night in the region, mostly with friends or family (52%) and in hotels (29%). Visitors to the area spent an average of $622 per person.

Moreover, a recent study commissioned by the Ottawa Festival Network found that visitors (excluding residents) attending annual festivals, special events, and fairs including Winterlude, Canada Day and CLAC, spent $122 million in 2016 during their stay in the city. While economic benefits are not an intended outcome of the CE Program, these results highlight the indirect impact of the program on, for example, the tourism sector in the CCR.

CIC Results

The CIC programming met its immediate expected result that Canadians have access to information and orientation to help them experience the sites and symbols of CCR. The number of requests for information from Visitor Services increased every year between 2013-14 (515,214) and 2017-18 (865,982). This includes interactions at the Capital Information Kiosk, with information officers along Confederation Boulevard, and through Discover the Hill and War Memorial Programs. It also includes e-mail and phone requests received through the Contact Centre.

The programming also met its expected intermediate result of CIC that Canadians experience CCR and its symbols and sites. The evaluation found that the number of youth that participated in tours in CCR did not change significantly, with an average of 26,701 tour participants annually. The Sound & Light Show on Parliament Hill was completely revamped and the new show, Northern Lights, was launched in the summer of 2015. Over the summer months the show, which makes use of state-of-the-art technology, sound, and lighting to tell Canada’s story, attracted nearly 270,000 people.

Monuments and public art are continually accessible and there is no current mechanism in place to track the number of visitors.

As well, Capital Experience received numerous external awards for their work over the period covered by the evaluation:

4.2. External Environment

4.2.1. External environmental challenges and opportunities

Evaluation question: What external environmental challenges and opportunities exist which affect the CE Program?

Various challenges and opportunities originating from the external environment have had an impact on the program, including strengthened security on Parliament Hill, involvement in the delivery and logistics related to the Canada 150 celebrations, unpredictable weather conditions, and changes to the NCC governance structure. Media coverage, whether positive or negative, can impact the program delivery.

CE Program experienced some external environmental challenges

Heightened security measures were implemented following a significant public security threat on Parliament Hill in October 2014. These new measures increased overall expenditures for the CE Program and affected visitor experience. According to the results of Canada Day public satisfaction surveys, there was a decrease in satisfaction with celebrations in 2017 compared to 2015. However, CCPO increased its security protocols while improving visitor experience based on lessons learned following enhanced security requirements and crowd control for their large-scale events.

Involvement of CE Program employees in the delivery of the Canada 150 celebrations affected their workload in 2017 across most programs: the Celebrate Canada period (June 21-July 1st), Winterlude, and the Sound and Light show. The number of interactions at the Capital Information Kiosk and through the various interpretation programs also increased significantly in 2017.

Weather conditions, particularly when these were suddenly unfavourable, negatively impacted uptake of CE Program events and activities as well as visitors’ satisfaction levels. In the context of climate change, variations in climate and local weather patterns are expected to have an ongoing impact on the CE Program.

Media coverage can impact program delivery including by influencing public perceptions and participation in events. According to the media content analysis, media coverage of events in CCR increased from 262 articles in 2013-14 to 365 articles in 2017-18. This would indicate increased media scrutiny and public opinion with respect to the delivery of program activities, though most media coverage (75%) was positive. Table 7 provides an overview of the sub-sample of stories identified by the media content analysis as reporting sentiments regarding CE Program activities, categorized by event or activity type.

Table 7: CE Program media content analysis, sample, 2013-14 to 2017-18
Stories by sentiment about activities and events, search topic, and regions Positive (number) Negative (number)
Events – Winterlude 17 3
Events – Canada Day 11 2
Events – Christmas Lights Across Canada 11 3
Art Installations 8 0
Monuments 8 9
Interpretation 4 1
Transition from NCC to Canadian Heritage 3 3
Totals 62 (75%) 21 (25%)

Source: Heritage CEP Media content analysis Technical Report

4.2.2. Program response to challenges from the external environment

Evaluation question: How has the Program responded to these challenges and opportunities?

The program implemented communications and risk management strategies to mitigate the external challenges. Specific steps taken included, but were not limited to, developing communications strategies for managing the impact of adverse weather conditions and providing media training for employees to better navigate public opinion of events and activities. The program worked with security partners, including the Parliamentary Protective Service for its events on Parliament Hill, to develop and put in place security plans to ensure the safety and security of event attendees. However, there may be opportunities for the CE Program to improve further its collaboration and communication with external stakeholders on operational and programming matters.

The CE program undertook several strategies to manage the effects of external environmental factors

KIIs and CE Program documents indicated that the CE Program implemented effective risk mitigation strategies to manage the effects of factors from the external environment. The most important mitigations were related to public security and its impact on planning, financial resources, and crowd management. Specifically, the program developed and implemented specific risk management strategies for each major event, such as Winterlude and Canada Day. KIIs reported that the program managed the safety and security of events in close collaboration with the program’s many security partners.

The program is aware of the need to address the effects of climate change, which impacts participation and experience. The document review noted the existence of contingency plans for program delivery, with solutions such as changing delivery dates or the type and location of events. Key informants also noted strategies were in place to manage the effects of changing and adverse weather conditions. These included a risk management strategy with contingency plans to deal with certain unforeseen events related to the weather, such as extreme heat during the summer months, as well as media training offered to staff to assist with dealing with public inquiries.

There are opportunities for the CE Program to further address external environmental factors

According to key informants and review of program documents, the program could continue and further address external factors through strengthened partnerships and additional risk management. KIIs identified the positive results of the department’s more recent participation in the Ottawa and Gatineau tourism committees. Other partnership opportunities include further development of the community partner-based approach taken for the delivery of Winterlude 2019.

At the time of the transfer, the NCC reported to the Minister of Canadian Heritage. There was some suggestion that this reporting relationship created a perceived conflict of interest for PCH participation in committees with external stakeholders, including the NCC. However, in 2016, the NCC changed their governance structure, and in 2019 was transferred out of the PCH portfolio to Public Services and Procurement Canada. This represented new opportunities for the program to engage with the NCC and other external partners, such as the mayors of Ottawa and Gatineau, to develop strategic relationships and collaborations.

4.3. Mission and strategy

4.3.1. Mission and strategy: understanding of program

Evaluation question: Do program representatives and relevant stakeholders have a clear understanding of the objectives and mandate of the CE program?

Although the mandate and the objectives of the transfer are clearly articulated in multiple official documents, internal and external stakeholder understanding was divided. Half of those interviewed indicated that the mandate of CE that focuses on the promotion of the Capital is unclear. Nonetheless, there was some improvement in MECCE employee satisfaction scores on whether the sector clearly expresses its vision, mission, and objectives, as noted in the PCH Employee Survey.

The objective of the CE program transfer and its mandate were consistent in documentation

Several program documents outlined the objectives and mandate of the CE Program post-transfer. For example, the 2013 Economic Action Plan Act stated that the purpose of the transfer was to ensure that a perspective was brought to celebrations and that events draw on the cultural and social fabric of the whole of Canada.Footnote 6 Additionally, the MoU developed between the NCC and PCH when the program was transferred stated that its objective was to ensure that “ongoing activities of the CE Program are maintained without interruption as of the September 30, 2013 transfer.”Footnote 7

The Department of Canadian Heritage Act was amended in 2013 to include the CE Program. The Act has the following paragraph added, under the description of the Minister’s powers, duties and functions: (k.1) the organization, sponsorship and promotion of public activities and events, in the National Capital Region as defined in section 2 of the National Capital Act, that will enrich the cultural and social fabric of Canada, taking into account the federal character of Canada, the equality of status of the official languages of Canada and the heritage of the people of Canada.

Key informants provided mixed feedback on mandate clarity

Program representatives and stakeholders interviewed were divided on whether the mandate of the CE Program was clear following the transfer. While about half of the CE Program staff and external partners interviewed indicated that the mandate was clearly defined, there were also questions on whether promotion of the Capital remained an objective now that the CE Program was in a department with a national mandate.

4.3.2. Mission and strategy: alignment with program mission, mandate, objectives and priorities

Evaluation question: To what extent are the CE program mission, mandate, objectives and priorities, as currently stated, aligned with:
- the PCH DRF, core responsibilities, strategic outcomes and PCH and Government of Canada priorities;
- SMEC mission, mandate, objectives and priorities; and
- MECCE mission, mandate objectives and priorities?

The objectives and mandate of the CE program, including its two key components, clearly align with the mandate and priorities of PCH and the Government of Canada, MECCE, and SMEC. The program supports PCH’s strategic outcomes related to Canadians’ engagement in celebrations and commemorations of national significance, and to ministerial priorities of diversity and inclusion. The program is well aligned with MECCE’s mandate to strengthen Canadians’ sense of connection to each other and to Canada. Further, CE Program contributions were well articulated in PCH performance reports. The CE Program thus benefited from good visibility in relation to the alignment of its activities with government priorities, particularly with respect to the sharing, expression, and appreciation of the Canadian identity.

Program mandate, objectives, and priorities aligned with PCH DRF, PCH core responsibilities and strategic outcomes, and PCH and Government of Canada priorities

The objective of the CE Program is to foster feelings of pride and belonging among Canadians towards Canada’s Capital, while increasing overall awareness of CCR as a destination where people can experience Canada’s heritage, culture, and achievements. The CE Program aims to ensure that Canadian values, stories, and symbols are represented in its activities and events.

In 2017-18, the CE Program was integrated into the DRF under Core Responsibility 2 – Heritage and Celebration and its objective supported the result “Canadians are engaged in celebrations and commemorations of national significance.” Prior to the creation of the DRF, the CE Program was also well aligned with the Department’s strategic outcome, “Canadians share, express, and appreciate their Canadian identity”, which was previously part of the Program Activity Architecture (PAA).

It was aligned with the government priority of diversity - highlighted in the 2015 Speech from the Throne – as the program aimed to showcase Canada’s diversity throughout its celebrations, interpretation, commemorations and public art.Footnote 8 Further, the program is currently exploring how and to what extent it can contribute to Sustainable Development Goals (SDG).

Program mandate, objectives, and priorities aligned with mission, mandate, and objectives of SMEC and MECCE

The CE Program is well aligned with the mission, mandates, objectives and priorities of the sector and branch charged with its delivery, SMEC and MECCE. It is aligned with the MECCE branch mission of promoting active participation, recognizing the common values of Canadians and celebrating the diversity of people and events that shape Canada. MECCE plays a leading role in carrying out interpretation, commemoration and awareness-raising activities and programs aimed at the general public, the promotion of sites and symbols of national importance, as well as efforts to promote the Capital as a place of pride and discovery for all Canadians.

The CE Program objective also supported the overall mandate of MECCE to strengthen Canadians’ sense of connection to each other and to Canada, advance awareness of and appreciation for Canadians’ shared values, cultural diversity, symbols, and institutions, and provide opportunities for Canadians to engage in public commemorations and celebrations of Canadian events and accomplishments.

4.3.3. Mission and strategy: support for Equity-Deserving Groups

Evaluation question: Does the CE Program support equity-deserving groups?

The CE Program demonstrated support for cultural diversity, as well as government and PCH priorities for equity-deserving groups. There was evidence that internal department policies and practices include consideration of diverse and inclusive hiring practices. The activities and events delivered by the programming highlighted culturally and linguistically diverse artists and paid particular attention to Indigenous culture.

Internal policies and procedures supported diversity

Analysis of CE Program performance data demonstrated that existing policies and procedures reflect Canada’s diversity. For example, 25% of the volunteers involved in CE Program events and activities were born outside of Canada; this compares to the 21.9% of Canada’s population that is born outside of Canada according to the 2016 Census.

CE Program events and activities promoted diversity

The majority of KIIs agreed that diversity and inclusion were well integrated into the program. KIIs emphasized the strong involvement of artists from diverse groups, including Indigenous and racialized peoples and LGBTQ2+, in the delivery of activities, particularly during Canada Day and Winterlude. For example, the Canada Day report (2017) noted that 700 artists were invited and that the program made significant efforts to reflect Canada's cultural diversity by involving artists from various linguistic and cultural backgrounds. Some interviewees also emphasized that Canada Day had increased emphasis on regional diversity, Indigenous cultures, and multiculturalism. Other KIIs cited Winterlude as a key contributor to diversity efforts, as it consistently incorporated Inuit games among its activities, as one example.

Documents show CIC and partners mounted temporary public exhibits featuring Anglophone and Francophone artists, and, in 2016 and 2017, specifically highlighted Indigenous and female artists.

The media analysis provided evidence of how programming highlighted cultural diversity. Canada’s cultural diversity was addressed in 31% of stories analyzed, and related to monuments, Canada Day, and CLAC. Monument stories, including those related to the Holocaust or escapes from communism, often referred to multiple generations of Canadians of various origins and the struggles faced by immigrants. There was also recognition of Indigenous peoples, for example with the erection of a monument to 17th century Algonquin leader Chief Tessouat, considered to be “very symbolic and very important to the Anishinabeg community.”Footnote 9

4.4. Organizational culture

4.4.1. Organizational culture: CE integration with PCH and MECCE

Evaluation question: How effectively has the CE Program integrated with PCH and MECCE?

There is evidence that the integration of the CE Program within PCH and MECCE has been mostly effective. There was successful continued delivery of events and projects throughout and after the transfer. Though there were initial challenges with the integration, KIIs noted that the situation had stabilized over time.

The integration of the program was mostly successful

The successful continued delivery of events and activities is evidence of successful integration of the programming within PCH. Employee feedback has also been mostly positive.

The results of the 2017 PSES demonstrated that the majority of CE Program employees had resettled into new working circumstances following the transfer of the CE Program from the NCC to PCH (more in section 4.7). Several KIIs confirmed that integration was generally successful despite some challenges experienced during the initial transition period.

The results of the document review found some examples of actions undertaken to facilitate the integration of the CE Program into PCH. For example, the Human Resources Plan (2015) of the MECCE Amalgamation of Branches stated that that process could be an opportunity to address outstanding issues, and update and evaluate staff positions in their new organizational structure.

4.5. Structure

4.5.1. Structure: Effectiveness and efficiency of current design and delivery model

Evaluation question: Is the current CE Program design and delivery model the most effective and efficient for the delivery of the Program’s mandate?

While the program’s design and delivery model offers features that support efficiency, the evaluation identified some opportunities for improvement. The manner in which CE Program delivery is organized inside MECCE has led to collaborations among programs, mostly with respect to event delivery. Consistent with its outward-facing role, the CE Program has many partnering arrangements and collaborations at the project level, both internal and external to PCH, which has facilitated program delivery. Additional efficiencies could be found through clarification of some roles and responsibilities and to ensure a common approach for strategic communications and collaborations.

The integration of the CE Program model into PCH was generally complete at the time of the evaluation, but some challenges remained in terms of adapting the business-oriented model of the NCC to the realities of a federal department. Specifically, the CCPO components were based on a model with business lines that retained decisional flexibility, contracting, and sponsorship flexibility.

As a result of the difference between this model and the operational requirements of the federal government context, KIIs identified some obstacles with procurement, sponsorship, and communications. Procurement challenges included lack of role clarity, understaffing of procurement and subsequent delays in processes, and different finance and contracting procedures (for example, many levels of approval required). However, several changes, particularly communications, have since been implemented that have improved these elements of CE Program delivery.

There was complementarity between the CE Program and other MECCE activities

Although there were similarities between the CE Program and other MECCE activities, several KIIs stated that there was no obvious or intentional duplication of efforts between the two bodies. In fact, the 2017 Performance Information Profile demonstrated complementarity between the CE Program and MECCE’s State Ceremonial and Canadian Symbols program (which complements the CE Program as it manages national ceremonies, including State funerals, installations of new Governors General, and the protocol element of the Canada Day Noon Show) and the Celebration and Commemoration Program (which complements the CE Program as it funds the delivery of Canada Day outside the Capital region and commemorations of national scope through grants and contributions).

There was evidence of collaboration between the CE Program and MECCE. For example, most KIIs recognized and appreciated the Collaborations and Synergies document created in 2017 which outlined the many potential collaborations and synergies among the five MECCE units (i.e., CE Program and MECCE combined), including in the delivery of events, volunteer services, programming, sponsorship and partnership development, production services, broadcasting, and Visitor Services.

Roles and responsibilities were not clearly defined in the MoU between the NCC and PCH

The MoU governing the relationship with the NCC listed areas of continued shared responsibility between the NCC and PCH, as well as areas of distinct responsibility. It referred frequently to coordination and consultation between the NCC and PCH with regards to the CE Program, with 19 appendices to govern this relationship. PCH was the overall project manager for the CE Program, while the NCC supported PCH in certain areas (for example, design and site selection) and managed and maintained NCC lands and associated structures.

The MoU stated that NCC and PCH would each manage their own volunteers, sponsorships, interpretation activities, and tours but responsibilities were divided for some programs and activities (for example, Flag/Banner Program, National Celebrations on NCC land). CIC retained most of the responsibility for programs that were managed jointly by PCH and NCC, such as commemorations. Where projects had joint responsibilities, PCH retained overall project management responsibilities. PCH was also responsible for policy, planning and design, while the NCC was generally responsible for construction, maintenance, and ongoing costs.

However, the evaluation identifies some gaps in understanding of roles and in communication. No revision date was set in the MoU, though it stated that both parties agreed to “jointly develop annual plans for priority projects with a three-year outlook.” No such plans were available at the time of the evaluation and there was no evidence that a review exercise was taking place. Some key informants suggested needs to clarify and further detail operational processes and lines of communication between PCH and NCC, which was transferred to the PSPC portfolio in November 2019. Finally, staff turnover and the subsequent loss of corporate knowledge amplified uncertainties and further underscored the need for enhanced clarity regarding the respective roles and responsibilities of PCH and the NCC.

Existing collaborations were identified but there are further opportunities to enhance synergies within PCH, between PCH and the NCC, and between PCH and other external partners

The majority of KIIs identified the existence of formal and informal communication and collaboration mechanisms with, for example, various tourism committees, the cities of Ottawa and Gatineau, and the Library of Parliament. PCH and the NCC collaborated for activity and event delivery, particularly for one-time projects such as monuments, which each had a working committee co-chaired by the sponsoring stakeholder and Assistant Deputy Minister (ADM). However, most KIIs reported that the relationship between the NCC and PCH would benefit from more regular and strategic communication efforts, and a few KIIs suggested the creation of a committee of all federal government agencies engaged in similar efforts in CCR. To support these efforts, several KIIs suggested the idea of negotiating a new protocol agreement that could include having a senior official designated to maintain communication with their counterpart at the NCC.

4.5.2. Structure: Alternative delivery models

Evaluation question: Are there alternative delivery models that could be considered?

Based on a review of models used in other major cities, there are alternative delivery elements that could be considered for the delivery of programming. Some features worthy of further exploration include major event delivery by multiple agencies and the use of external funding.

Four alternative delivery models offer considerations for program delivery

From an examination of the delivery models used by similar programming offered in four capital cities, Washington, D.C., Canberra, Australia, Quebec City, Quebec, and Regina, Saskatchewan, the evaluation identified a number of promising practices. All four capitals maintained advisory committees of experts on commemorative themes and public art. There are formal expert committees for the validation and approval of commemorative themes, monuments, and public art in Canberra and Washington, D.C. (inscribed by law), and Quebec City. This was also part of the NCC model for programming but has not been carried over to the CE model. Since the transfer of CE, there is no longer a formal advisory committee and advice has been sought on an ad hoc basis.

With respect to finances, some jurisdictions rely on external funding to facilitate event and activity delivery. For example, the EventACT agency in Canberra must contribute to its own income, including through ticketing and licenses.

There was no strong support among KIIs for the implementation of a new delivery model for the programming, although some noted value in exploring revenue diversification. Some of the KIIs reported that alternative delivery models would not necessarily improve the effectiveness or efficiency of the CE Program. For example, they did not foresee any benefits to integrating the private sector into program delivery, and a few indicated that privatization could result in loss of emphasis on government priority messages.

4.6. Systems

4.6.1. Systems: Unintended impacts on program delivery

Evaluation question: What are the unintended impacts on delivery of the CE Program transfer to a federal department?

Major events were successfully delivered; however, the transfer may have led to more last-minute work, and additional workflow implications due to the policy and regulatory differences between a Crown corporation and a federal department. These affected the extent to which sponsorships could be used for events, and the turnaround times for the hiring of artists for major events.

KIIs reported more delays in approvals and contracting due to federal environment

KIIs reported that delays in approvals, more complex departmental planning and budgeting cycles, and stricter rules regarding contracting and hiring practices may have had the unexpected effects of creating more last-minute work and increased overtime. Specifically, delays in planning caused by more extensive and rigid procedures for approvals meant that tasks were often performed at the last minute, particularly in procurement.

The transition of the CE Program from a Crown corporation to a department had an impact on staff and workflow. Several KIIs noted that there were additional levels of approval to navigate for decision-making following the transfer, which prolonged processes and reduced efficiency. They also reported that the CE Program faced restrictions in relation to the extent to which sponsorships could be used for events, which was a significant financial obstacle.

Challenges related to sponsorship are further detailed in the following section (4.6.2).

4.6.2. Systems: Impact of Program resources on achievement of outcomes

Evaluation question: To what extent do program resources (such as human and financial) enable the effective delivery of the CE Program activities and the achievement of its outcomes?

The transfer of the CE Program from the NCC to PCH had impacts on the delivery and results of the programming, particularly in the beginning. Transferred HR resources (FTEs) were not fully available to CE for program delivery. Eighty-one FTEs were transferred from the NCC to PCH on September 30, 2013. Just under two thirds of indeterminate and term FTEs were assigned to CCPO or CIC, while one third were transferred to other areas within PCH, such as communications and finance. Sponsorship faced funding gaps and barriers related to the misalignment of the federal sponsorship model to the former model at the NCC, however by 2017-18, revenue had met goals. While CE events and projects were generally delivered as planned, these HR resource factors had implications for the efficiency of CE Program delivery.

Impact of the allocation of HR resources upon transfer to PCH

Approximately 81 FTEs were transferred from the NCC to PCH on September 30, 2013. According to PCH human resource data, the CE Program received 52 indeterminate FTEs as well as 4 non-indeterminate FTEs with the transfer. The program also had 13 part-time students at the time of transfer. Six other units received a total of 24 indeterminate FTEs from the NCC transfer, with the largest proportion, 11 FTEs, going to corporate communications.

KIIs stated that they felt that they were not receiving support from Internal Services equivalent to the 24 FTEs transferred to those areas, as those FTEs were assigned to other work within PCH, rather than being specifically assigned to CE. However, managers reported improved services towards the end of the evaluation period.

Sponsorship met its goals despite initial barriers

The NCC business model for major events depended on revenues and in-kind contributions raised through corporate sponsorship. At the time of the transfer, the TBS transferred $2M from PCH’s annual appropriations into a vote-netted account to finance and benefit from corporate sponsorships. CE Program revenues from sponsorship and other sources surpassed the goal of $1.16M for sponsorships in 2015-16 (final surplus of $414,000) and 2017-18 (final surplus of $86,000).

Although the 2017 celebrations saw a significant increase in sponsorship, with the concomitant revenue increase, years following the evaluation period have offered challenges. Sponsorship in the context of a federal department can create complexity for CCPO and PCH Communications. Further, large investments made for 2017 meant businesses often reduced their sponsorship allotments for subsequent years. Finally, the virtual events necessitated by the pandemic increased sponsorship challenges in recent years.

5. Conclusions

The Burke-Litwin model enabled the examination of factors related to organizational change, and the impact of the transfer of the program from the NCC to PCH. Overall, the findings highlight the importance of change management best practices including strong internal and external communications and of strengthened partnerships.

Although not the focus of the evaluation, there is evidence that the programming achieves or contributes to expected results through the delivery of events, programming initiatives and visitor services. The programming increases access to events and activities in CCR and nationally via media, that celebrate Canadian identity and reflect Canada’s diversity. It delivers services that improve the availability of information and orientation that helps visitors experience the sites and symbols of CCR.

The evaluation demonstrated that the mandate and objectives of the CE Program and its two components (i.e., CCPO and CIC) are clearly aligned with the PCH DRF and the main responsibilities, mandate, and objectives of the MECCE branch. It was also well aligned with federal government priorities, particularly the promotion of diversity through its celebrations, commemorations, and public art.

The purpose of the CE Program’s transfer to PCH was to ensure that, moving forward, a “broad, national perspective is brought to celebrations, and that events such as Winterlude and Canada Day represent the cultural and social fabric of the whole of Canada.” While the objective and mandate of the transfer was articulated and communicated in multiple official documents, half of KIIs interviewed indicated that the mandate was not clear, specifically whether the mandate to promote the Capital remained.

The integration of the CE Program model into PCH was generally complete at the time of the evaluation, but there remained some challenges in terms of adapting the CE Program model to the realities of a federal department. This was particularly evident for procurement and communications processes, which entailed greater flexibility at the NCC than at PCH. However, strategies implemented since the completion of the evaluation resolved some of these concerns.

Specifically, it should be noted that between the end date of the evaluation period and the publication of this evaluation report, the relationship between the CE Program and the PCH Communications team has continued to evolve and strengthen. For CCPO, a new approach was established in 2019 where Communications now has a dedicated team of marketers and communication specialists assigned to CCPO to ensure continuity in services and expertise. CIC has worked closely with its communications colleagues to develop an integrated approach to promotion. Communications colleagues take part in weekly program meetings to identify opportunities for promotion and outreach, especially on key priorities, and to enhance the program’s visibility.

Significant progress has also been made since the end of the evaluation period in the areas of security and crowd management. Since 2017, CCPO has developed a crowd management expertise within its directorate, completing a thorough analysis of event security posture by consulting experts in the field of crowd management. CCPO implemented measures to reduce the public’s wait time and established a process with security partners to improve visitor experience, resulting in a significant reduction of wait time for Canada Day 2018.

There is no apparent duplication of the program offering, although similar activities are delivered by several MECCE units. The CE Program had formal and informal partnerships with the NCC related to tourism and formal partnerships for the delivery of many projects, such as monuments. There was evidence of the need for further clarification of the roles and responsibilities in the MoU, particularly the operational processes between the NCC and PCH. The majority of KIIs believed that the relationship between the NCC and PCH would benefit from more regular and strategic communications, and that the program could be improved by enhancing relationships with other external partners (for example, the cities of Ottawa and Gatineau).

6. Recommendations, management response and action plan

The evaluation recognizes that many of the challenges related to the transfer of the CE program from the NCC to PCH have been addressed since this project was launched. The program has done considerable work since the period of the evaluation to support the integration of CE into the department and the continued achievement of results. Therefore, and given the findings and conclusions, the evaluation identifies one broad area of recommendation: to further develop and communicate and implement strategic priorities to guide and define the program’s development and delivery moving forward.

Recommendation 1

The evaluation recommends that the assistant deputy minister for Sport, Major Events and Commemorations Sector further develop, communicate and implement strategic priorities for the program, which will guide and define the program’s development and delivery.

Management response

The Major Events, Commemorations and Capital Experience Branch (MECCE) accepts this recommendation, acknowledges the report findings, and agrees that the areas related to the transfer have been largely addressed. While the transfer of Capital Experience (CE) from the National Capital Commission (NCC) presented challenges, much has been resolved and additional progress has been made since the evaluation period ended. In addition to the areas mentioned in the report, progress has been made in the following specific areas since 2017-18:

National Vision: While CE has consistently maintained a national perspective in its programming, the pandemic pushed the program to expand the national reach of events. Canada Day 2020 offered, for the first time ever, a completely virtual Canada Day celebration which included two national broadcasts with programming from cities across the country, as well as enhanced online content. The same national virtual reach occurred for Winterlude in 2021. National planning and delivery were strengthened by working with a broad interdepartmental framework via the Interdepartmental Commemoration Committee, using whole-of-government anniversaries to guide new segments for the Sound and Light Show, temporary exhibits and thematic Confederation Boulevard banners.

Partnerships: In 2019, CE reviewed Winterlude to ensure it remained a vibrant winter festival in Canada’s Capital Region, which resulted in a more urban approach with an increase in strategic partnerships. For example, the 2019 edition of Winterlude saw a significant shift of programming and activities being offered in partnership with Business Improvement Areas (BIAs).

As the focus has clearly shifted away from the transfer of the program to Canadian Heritage from the NCC, timing is right to focus attention on developing strategic priorities to guide a successful future. The development of these priorities will help to clarify roles and responsibilities while informing strategic partnerships and effective delivery that respond to the needs of Canadians. MECCE understands that effective and timely implementation of the Action Plan is necessary to ensure sustained program improvement.

Table 8: recommendation 1 – action plan
Action Plan Item Deliverables Timeline Responsible
1.1 Hold initial planning meetings to situate CE within current MECCE activities and identify common challenges and potential synergies 1.1.1. Initial meetings with internal stakeholders and summary meeting notes with next steps March 31, 2022 Director General (DG), MECCE with support from all directorates
1.2. CE to articulate short- and medium-term priorities 1.2.1. Approved list of priorities and strategic plan December 31, 2022 DG, MECCE; Directors, CE
1.3. CE to communicate strategic priorities to employees and relevant partners 1.3.1. DG/Directors-level meeting(s) with employees, and with stakeholders and partners to discuss priorities and articulate steps for future collaboration March 31, 2023 DG, MECCE; Directors, CE

Full implementation date: March 31, 2023

Annex A: Evaluation framework

The evaluation framework below presents the evaluation questions, indicators, and data collection methods that were used for the evaluation.

External environment: Question 1.1. What external environmental challenges and opportunities exist which affect the CE Program?
Indicators Data sources Methods of data collection
  • Evidence of external factors and their impact on the Program
  • Program management and staff
  • Internal stakeholders
  • External stakeholders
  • Program documents
  • Media publications
  • Interviews
  • Document review
  • Media content analysis
External environment: Question 1.2. How has the CE Program responded to these challenges and opportunities?
Indicators Data sources Methods of data collection
  • Evidence of effective risk mitigation strategies
  • Evidence of the Program’s response to emerging challenges and opportunities and of ongoing efforts to enhance the Program
  • Perspectives on the extent to which CE Program activities currently address emerging challenges and opportunities, ongoing initiatives to enhance the program, and opportunities to do so more effectively
  • Program management and staff
  • Internal stakeholders
  • External stakeholders
  • Program documents
  • Interviews
  • Document review
Mission and strategy: Question 2.1. Do Program representatives and relevant stakeholders have a clear understanding of the objectives and mandate of the CE Program, following its transfer from the NCC to PCH?
Indicators Data sources Methods of data collection
  • Evidence that the objective of the CE Program transfer and its mandate under PCH were clearly outlined in relevant documents (for example, TB subs, Budget, MoU, DPRs, RPPs, PIPS, Department of Canadian Heritage Act, National Capital Act, etc.) and communicated
  • Perspectives on the clarity of the objective and mandate of the CE Program transfer, and the current mandate of the Program
  • Program management and staff
  • Internal stakeholders
  • External stakeholders
  • Program documents
  • Media publications
  • Interviews
  • Document review
  • Media content analysis

Mission and strategy: Question 2.2. To what extent are the CE Program mission, mandate, objectives, and priorities, as currently stated, aligned with:

  • The PCH DRF, core responsibilities, strategic outcomes and PCH and GoC priorities;
  • Sport, Major Events, and Commemorations Sector mission, mandate, objectives and priorities; and
  • MECCE mission, mandate, objectives and priorities.
Indicators Data sources Methods of data collection
  • Alignment of the current stated objectives, mandate, and activities of the CE Program with PCH departmental outcomes and priorities, and with Sector and Branch missions, mandates, objectives and priorities, as stated in PCH corporate documents (for example, DPRs, RPPs, etc.)
  • Perspectives on the level of integration of the CE Program within the PCH, the Sector and Branch
  • Evidence of the visibility of the Program externally and within PCH corporate documents (for example, DPRs, RPPs, etc.)
  • Program management and staff
  • Internal stakeholders
  • External stakeholders
  • Program documents
  • Administrative data
  • Media publications
  • Interviews
  • Document review
  • Administrative data review
  • Media content analysis
Mission and strategy: Question 2.3. Does the CE Program support Equity-Deserving Groups?
Indicators Data sources Methods of data collection
  • Evidence of CE Program policies and procedures and activities that promote diversity
  • Examples of how Program events and activities and interpretive content, commemorations and public art reflect Canada’s cultural diversity, Indigenous cultures and official languages
  • Participant and visitor demographics
  • Program management and staff
  • Internal stakeholders
  • External stakeholders
  • Program documents
  • Administrative data
  • Media publications
  • Interviews
  • Document review
  • Administrative data review
  • Media content analysis
Organizational culture: Question 3.1. How effectively has the CE Program integrated with PCH and MECCE?
Indicators Data sources Methods of data collection
  • Perspectives on the effectiveness of the integration of the CE Program with into PCH and MECCE
  • Evidence of actions taken to integrate CE with PCH and with MECCE
  • Program management and staff
  • Internal stakeholders
  • Program documents
  • Interviews
  • Document review
Structure: Question 4.1. Is the current CE Program design and delivery model the most effective and efficient for the delivery of the Program’s mandate?
Indicators Data sources Methods of data collection
  • Evidence of an understanding of roles and responsibilities among PCH stakeholders, including within the CE Program areas and MECCE, as well as in relation to corporate service areas, including procurement, communications, finance etc. to support program delivery effectively and efficiently
  • Evidence of mechanisms to facilitate communication and collaboration amongst internal stakeholders (for example, Committees, working groups, information sharing), and their perceived effectiveness
  • Perspectives on and expert opinion of the effectiveness of the current delivery model, ongoing initiatives to enhance the program, and opportunities for alternative delivery models
  • Perspectives on duplication of effort by CE Program staff and internal stakeholders
  • Evidence of adjustments made to existing mechanisms to better facilitate communication and collaboration among internal stakeholders
  • Perspectives on the perceived effectiveness of adjustments to existing mechanism and opportunities for synergies and integration within the CE Program and MECCE
  • Evidence that roles and responsibilities of the NCC and PCH are clearly outlined in the bilateral MoU
  • Existence of mechanisms to facilitate communication and collaboration among external stakeholders (for example, MoUs, committees, working groups, information sharing), and their perceived effectiveness
  • Perspectives on opportunities for further partnership/collaboration
  • Program management and staff
  • Internal stakeholders
  • External stakeholders
  • Program documents
  • Administrative data
  • Interviews
  • Document review
  • Administrative data review
Structure: Question 4.2. Are there alternative delivery models that could be considered?
Indicators Data sources Methods of data collection
  • Perspectives on alternative delivery models
  • Program management and staff
  • Internal stakeholders
  • External stakeholders
  • Interviews
Systems (policies and procedures, human resources and resource planning): Question 5.1. What are the unintended impacts of the CE Program transfer to a federal department on Program delivery?
Indicators Data sources Methods of data collection
  • Evidence of constraints and issues that hinder the efficient and effective achievement of the program mandate, objectives and activities
  • Alignment of timelines associated with internal processes, systems, and services with program planning cycles/timelines
  • Evidence of adjustments made to the design and delivery of the Program to account for internal constraints
  • Program management and staff
  • Internal stakeholders
  • Program documents
  • Administrative data
  • Interviews
  • Document review
  • Administrative data review
Systems (policies and procedures, human resources and resource planning): Question 5.2. To what extent do program resources (such as, human and financial) enable the effective delivery of the CE Program activities and the achievement of its outcomes?
Indicators Data sources Methods of data collection
  • Distribution of FTEs and funds following the program transfer
  • Percentage change in program expenditures by type, directorate (CIC vs. CCPO), and fiscal year
  • Planned HR (number of FTEs) and finances (budget) as compared to actuals, by directorate/fiscal year
  • Program expenditures by type, directorate (CIC vs. CCPO), and fiscal year
  • Planned # of FTEs, as compared to actuals, per directorate (CIC and CCPO), per fiscal year
  • Program documents
  • Administrative data
  • Document review
  • Administrative data review
Tasks and individual skills: Question 6.1. To what extent has the CE Program addressed human resource challenges?
Indicators Data sources Methods of data collection
  • Turnover rates, per directorate (CIC and CCPO), per fiscal year
  • Perspectives on skills gaps, recruitment issues, and staff mobility
  • Evidence of plans to address human resource challenges
  • Evidence of measures taken to address human resource challenges
  • Perspectives on the effectiveness of the Program’s response to human resource challenges
  • Program management and staff
  • Internal stakeholders
  • Program documents
  • Administrative data
  • Interviews
  • Document review
  • Administrative data review
Employees’ needs and values: Question 7.1. To what extent has the CE Program addressed the factors that affect employees’ work?
Indicators Data sources Methods of data collection
  • “Over to you” survey results
  • Rates of absenteeism, per directorate (CIC and CCPO), by fiscal year
  • Turnover rates, by directorate (CIC and CCPO), per fiscal year
  • Perspectives of Program staff on factors affecting their work
  • Evidence of measures taken by the CE Program to provide ongoing support to employees and to address the factors that affect their work
  • Perspectives of Program staff on the effectiveness of measures taken by the Program to provide ongoing support to address the factors that affect their work
  • Perspectives of Program staff of factors that would enhance their job experience
  • Program management and staff
  • Internal stakeholders
  • Program documents
  • Administrative data
  • Interviews
  • Document review
  • Administrative data review

Annex B: Capital Experience Program Logic Model

Appendix B: Capital Experience Program Logic Model – text version

CE Program Objective

To foster feelings of pride and belonging amongst Canadians towards their Capital, while increasing their overall awareness of Canada’s Capital Region as a destination where people can experience Canada’s heritage, culture, and achievements.

Program Outcomes

Long Term

  • Canadians are engaged in Celebrations and Commemorations of national significance. [DRF and PI result]
  • Canadians feel connected to Canada and each other.

Intermediate

  • Canadians participate in events and activities in Canada’s Capital Region that celebrate Canadian identity and reflect Canada’s diversity.
  • Canadians experience Canada’s Capital Region and its sites and symbols.

Immediate

  • Canadians have access to events and activities in Canada’s Capital Region that celebrate Canadian identity and reflect Canada’s Capital region.
  • Equipment and technical expertise are available for events in Canada’s Capital region.
  • Canada’s Capital Region features interpretive content, monuments, commemorations, and public art that reflect Canadian identity, culture, and values.
  • Canadians have access to information and orientation to help them experience the sites and symbols of Canada’s National Capital.

Reach

  • Canadian public (including across-Canada, via broadcasts and new media)
  • Participants (attendees/volunteers) in events and activities in Canada’s Capital Region
  • Not-for-profit events
  • Canadian public
  • Teachers / educators / students
  • Visitors to Canada’s Capital Region (monuments, public art, and commemorative art)

Outputs

  • Events and activities in Canada’s Capital Region (Winterlude, Canada Day, and Christmas Lights Across Canada Projection show)
  • Broadcast and new media related to events and activities in Canada’s Capital Region
  • Rent and Loan Program
  • Volunteer services at events and activities in Canada’s Capital Region
  • Monuments
  • Public art
  • Commemorative art
  • Interpretive content (Sound and Light Show on Parliament Hill, Discover the Hill, War Memorial Program, Roaming Services, Capital Region Tours, pedagogical tools)
  • Capital Information Kiosk and Visitor Services for sites and symbols (ex. Maps, officers)

Activities

  • Develop programming for events and activities in Canada’s capital Region.
  • Manage volunteer services at events and activities in the Capital Region.
  • Provide technical and logistical support in the production of events, activities, and shows in Canada’s Capital Region (production services).
  • Provide technical and logistical support in the production of events, activities, and shows in Canada’s Capital Region (production services).
  • Provide support to local not-for-profit events and manage crown assets housed in the PCH warehouse.

Capital Celebrations and Program Operations

  • Facilitate the development of new commemorations, public art, and monuments, working closely with the NCC and other key stakeholders (such as, PSPC).
  • Manage visitor services at the Capital Information Kiosk and at nationally significant locations in Canada’s Capital Region.
  • Develop interpretive content for monuments, public art, and commemorations in Canada’s Capital Region.
  • Establish contracts, agreements, and partnerships related to monuments, public art, and commemorative art.

Capital Interpretation and Commemorations

Inputs

  • Financial and Human Resources
  • Equipment and Supplies

Note: The pink text indicates that the objective also applies to the other division, as linked by the arrow

Annex C: Capital Experience Program Organizational Chart

Annex C: Capital Experience Program Organizational Chart– text version
  • Deputy Minister
    Canadian Heritage
    • Senior Assistant Deputy Minister
      Sport, Major Events, Commemorations and Portfolio Affairs
      • Director General
        Major Events, Commemorations and Capital Experience
        • Director
          Capital Celebrations and Program Operations
          • Manager
            Volunteer Services
          • Manager
            Broadcast and New Media
          • Senior Manager
            Sponsorship and Business Development
            • Manager
              Sponsorship and Business Development
          • Senior Manager
            Programming
            • Manager
              Event Coordination
            • Manager
              Event Coordination
          • Senior Manager
            Production Services
            • Manager
              Logistics
            • Manager
              Technical Services
        • Director
          Capital Interpretation and Commemorations
          • Senior Manager
            Monuments and Public Art
          • Senior Manager
            Capital Interpretation
            • Manager
              Visitor Programs
            • Manager
              Program Coordination

Annex D: Burke-Litwin model

A visual of the Burke-Litwin model and the dimensions of the model which were applied in the evaluation of the CE Program are described below.

Annex D: Burke-Litwin model
Annex D: Burke-Litwin model – text version
  • External Environment
    • Mission and Strategy
    • Leadership
    • Organizational Culture
  • Mission and Strategy
    • External Environment
    • Leadership
    • Organizational Culture
    • Structure
    • Work Unit Climate
    • Task Requirements and Individual Skills/Abilities
  • Leadership
    • External Environment
    • Mission and Strategy
    • Organizational Culture
    • Structure
    • Management Practices
    • Systems (policies and procedures)
  • Organizational Culture
    • External Environment
    • Mission and Strategy
    • Leadership
    • Systems (policies and procedures)
    • Work Unit Climate
    • Individual Needs and Values
  • Structure
    • Mission and Strategy
    • Leadership
    • Management Practices
    • Systems (policies and procedures)
    • Work Unit Climate
    • Task Requirements and Individual Skills/Abilities
  • Management Practices
    • Leadership
    • Structure
    • Systems (policies and procedures)
    • Work Unit Climate
  • Systems (policies and procedures)
    • Leadership
    • Organizational Culture
    • Management Practices
    • Structure
    • Work Unit Climate
    • Individual Needs and Values
  • Work Unit Climate
    • Mission and Strategy
    • Organizational Culture
    • Structure
    • Management Practices
    • Systems (policies and procedures)
    • Motivation
  • Task Requirements and Individual Skills/Abilities
    • Mission and Strategy
    • Structure
    • Motivation
  • Motivation
    • Work Unit Climate
    • Task Requirements and Individual Skills/Abilities
    • Individual Needs and Values
    • Individual and Organizational Performance
  • Individual Needs and Values
    • Organizational Culture
    • Systems (policies and procedures)
    • Motivation
  • Individual and Organizational Performance
    • Motivation
Annex D: Burke-Litwin model dimensions
Dimensions of Model Description
External Environment This includes such factors as markets, legislation, competition and the economy. All of these will have consequences for organizations.
Mission and Strategy An organization’s mission articulates its reason for existing. It is the foundation upon which all activity should be built. The strategy then sets out, in broad terms, how the organization will go about achieving its mission. Often, the strategy will be developed in light of environmental change and will have a significant impact on the organization’s work.
Leadership This considers the attitudes and behaviour of senior colleagues and how these behaviours are perceived by the organization as a whole. The way in which change is implemented and accepted through the organization will be largely influenced by the top team.
Organizational Culture Organizational culture can be described as “the way we do things around here”. It considers the beliefs, behaviours, values and conventions that prevail in an organization. Culture change does not happen overnight. It evolves over time as a result of many other changes in the organization.
Structure Very often, changes in strategy can lead to changes in the way the organization is structured. This can impact on relationships, responsibilities and ways of working.
Management Practices Considers how managers use human and material resources to carry out the organization’s strategy, and the style of management and their relationship to subordinates.
Systems (policies and procedures) Relates to an organization’s policies and procedures, including systems for reward and performance appraisal, management information, human resources and resource planning.
Work Unit Climate This considers employees’ perceptions of their immediate colleagues and working environment. Our immediate working environment is often what shapes our view of the organization as a whole and influences the extent to which we feel satisfied in our jobs. Changes to the immediate working environment need to be managed sensitively, as they are likely to invoke a range of emotional and political responses from staff. This is particularly the case where change involves moving location, a change in personnel, or a change in terms of conditions of service, such as working hours.
Task Requirements and Individual Skills/Abilities Change at a higher level in the organization will often require changes in the work carried out and the skills available in the team.
Motivation Considers the significance of individual and organizational goals. Motivation is key to effective change. The real challenge is to maintain motivation throughout a change project, particularly when change is often not well-received by those affected.
Individual Needs and Values Changes to team membership can mean a change in the team dynamic. In a perfect world, we would be able to recruit the exact fit for our teams, in terms of personal style, abilities and skills mix. However, in reality it is not always possible, and it is necessary to identify any risks in this area and mitigate them.
Individual and Organizational Performance Considers the level of performance in terms of productivity, customer satisfaction, quality.

Annex E: Media Analysis Strategy

1.0 Media Analysis Overview and scope

The analysis included French and English print and broadcast media sources on topics relating to the CE Program. These studied specific issues of the external environment in which the Program operates, including the mission and strategy of the Program, and others, as identified in the Evaluation Matrix. Specific research questions were developed based on the evaluation questions and areas where the media analysis could provide complementary evidence for the evaluation.

In order to narrow the scope to topics that would most likely to be covered in media publications, the media analysis focused on specific events, monuments, and public art installations overseen by the CE Program. In consultation with Program staff, specific recent and high-profile monuments and public art installations were identified for inclusion in the search terms and the analysis.

2.0 Sources for the media analysis

A preliminary assessment of MediaScope as a potential database source for the media analysis was conducted. However, due to the limited availability of data (i.e., only for 2017-18 and 2018-19), it was determined that an alternative database should be used for the analysis. ProQuest was therefore the main tool used for the media analysis, supplemented by online searches for select media sources. ProQuest is a global information content and technology company providing applications and products for libraries, with access to more than 3,000 of the world’s news sources.

Print and broadcast media sources from across Canada were consulted. The Canadian Newsstream database available through ProQuest was the primary source for the media analysis. This database includes 588 English and French online, print, and broadcast news sources from across Canada.Footnote 10 Specialized journals covering monuments and art displays were identified for inclusion by CE Program management and staff and will also be included in searches for relevant monuments and art installations (either through ProQuest or direct online searches):

3.0 Search criteria and sampling strategy

Articles were sampled from April 1, 2013 to March 31, 2018, which included the period leading up to and following the transfer of the CE Program from the NCC to PCH.

Search terms were developed to capture a sample of media coverage related to the CE Program transition from NCC to PCH, as well as specific relevant activities, events, monuments, and art installations. The search terms drew from a list of suggested search terms from CE Program staff and were adjusted to maximize the relevance of the search. Depending on the topic searched, one or more databases were used to ensure a more targeted approach (for example, monuments will be searched through the Canadian Newsstream and specialized journal media sources). For events, the sample was stratified by different fiscal years with the purpose of analyzing trends in media coverage over time, particularly in relation to Canada 150 which occurred in 2017-18 (pre-, during, and post-Canada 150, where applicable).

Annex F: Bibliography

©Her Majesty the Queen in Right of Canada, 2022
Catalogue Number: CH7-67/1-2022E-PDF
ISBN: 978-0-660-42299-2

Page details

Date modified: