Evaluation of the Capital Experience Program 2013-14 to 2017-18
Evaluation Services Directorate
January 6, 2022
On this page
- List of tables
- List of acronyms and abbreviations
- Executive summary
- 1. Introduction
- 2. Program profile
- 3. Approach and methodology
- 4. Findings
- 5. Conclusions
- 6. Recommendations, management response and action plan
- Annex A: Evaluation framework
- Annex B: Capital Experience Program Logic Model
- Annex C: Capital Experience Program Organizational Chart
- Annex D: Burke-Litwin model
- Annex E: Media Analysis Strategy
- Annex F: Bibliography
List of tables
- Table 1: program objective and expected outcomes
- Table 2: program spending, 2013-14 to 2017-18 (actuals)*
- Table 3: evaluation questions by Burke-Litwin dimension
- Table 4: limits and mitigations
- Table 5: CCPO Program Results, social media, 2014-15 to 2017-18
- Table 6: CCPO Program Results, viewership, 2014-15 to 2017-18
- Table 7: CE Program media content analysis, sample, 2013-14 to 2017-18
- Table 8: recommendation 1 – action plan
Alternate format
Evaluation of the Capital Experience Program 2013-14 to 2017-18 [PDF version - 1.3 KB]
List of acronyms and abbreviations
- ADM
- Assistant Deputy Minister
- CCP
- Celebration and Commemoration Program
- CCPO
- Capital Celebrations and Program Operations Directorate
- CCR
- Canada’s Capital Region
- CHIN
- Canadian Heritage Information Network
- CIC
- Capital Interpretation and Commemorations Directorate
- CE
- Capital Experience Program
- CLAC
- Christmas Lights Across Canada
- DGO
- Director General Office
- DRF
- Departmental Results Framework
- ESD
- Evaluation Services Directorate
- FAA
- Financial Administration Act
- FTEs
- Full Time Equivalents
- Gs&Cs
- Grants and Contributions
- GBA Plus
- Gender-Based Analysis Plus
- GoC
- Government of Canada
- HR
- Human Resources
- KII
- Key Informant Interview
- LGBTQ2+
- Lesbian, Gay, Bisexual, Transgender, Queer, Two-Spirited, Plus
- MECCE
- Major Events, Commemorations and Capital Experience Branch
- MoU
- Memorandum of Understanding
- NCC
- National Capital Commission
- O&M
- Operations and Maintenance
- PAA
- Program Alignment Architecture
- PCH
- Department of Canadian Heritage
- PIPs
- Performance Information Profiles
- PPS
- Parliamentary Protection Services
- PSES
- Public Service Employee Survey
- PSPC
- Public Services and Procurement Canada
- SMEC
- Sport, Major Events and Commemorations Sector
Executive summary
Program description
The Capital Experience (CE) Program supports the production of events and activities as well as the development and delivery of new monuments, public art, interpretation and information services in Canada’s Capital Region (CCR). CE supports such events as Winterlude, Canada Day celebrations, Christmas Lights Across Canada (CLAC), and the Sound and Light Show on Parliament Hill. The objective of the CE Program is to foster feelings of pride and belonging amongst Canadians towards their Capital, while increasing their overall awareness of Canada’s Capital Region as a destination where people can experience Canada’s heritage, culture and achievements.
Evaluation approach and methodology
This evaluation is the first for the program and it covers a 4.5-year period from 2013-14 to 2017-18. Based on input from program management gathered during scoping interviews and given that the program had been recently transferred from the National Capital Commission (NCC) to Canadian Heritage (PCH), the evaluation focused on questions related to organizational change.
The evaluation was guided by the Burke-Litwin Organizational Performance and Change Model. This model describes the linkages among the key factors that affect organizational performance and determine how change occurs within an organization. The core dimensions that were explored included: external environment; mission and strategy; organizational culture; structure; and systems. Methodologies used for the collection and analysis of data included document, administrative data, and literature reviews, key informant interviews (KII), and a media content analysis.
Findings
Program results
The evaluation team analyzed existing performance measurement data to provide an overview of the immediate and intermediate results over the period. Further, a media analysis was done to examine national perceptions of CE programming.
The immediate outcome of Canadians had access to events and activities in CCR that celebrate Canadian identity and reflect Canada’s diversity was met, as well as its intermediate outcome that Canadians participate in events and activities that celebrate Canadian identity and reflect Canada's diversity in the CCR. Attendance at events was generally stable over the evaluation period, with a high in 2017, due to the 150th celebrations.
The results of a 2015 satisfaction survey on Canada Day in the Capital indicated that a majority of respondents (71%) agreed that the activities were representative of Canada, and more than half of respondents (62%) agreed that their visit gave them a better appreciation of Canada's cultural and social diversity. In 2017, a poll conducted with funding from Canada 150 demonstrated that nearly two thirds of respondents (63%) agreed that the festivities were representative of Canada.
The CE programming also contributed to Canadians having access to information and orientation to help them experience the sites and symbols of CCR. The number of requests for information from visitor services increased every year between 2013-14 (515,214) and 2017-18 (865,982). The programming also met its expected intermediate result of Capital Interpretation and Commemorations (CIC) that Canadians experience CCR and its symbols and sites.
Over the period covered by the evaluation, the programming received numerous external awards. The program’s results were also demonstrated through media analysis and show that 75% of the media mentions of program activities were positive. There was an increase in media mentions over the period of the evaluation, from 262 articles in 2013-14 to 365 articles in 2017-18.
External environment
Various external challenges impacted the CE Program, including strengthened security on Parliament Hill, involvement in the Canada 150 celebrations, increased media attention to program events and projects, and more frequent adverse weather conditions. The CE Program implemented several strategies to address these factors and improve program delivery, including developing a strategy for public communications regarding weather conditions and providing media training to employees to manage the impact of increasing media attention to events and monuments. The program also increased its security protocols, while improving visitor experience, based on lessons learned from previous events.
Mission and strategy
The CE objectives are well aligned with PCH’s mandate and priorities, particularly with respect to diversity and inclusion and strengthening Canadians’ sense of connection to each other and to Canada. It supports the overall mandate of PCH’s Major Events, Commemorations and Capital Experience Branch (MECCE), to strengthen Canadians’ sense of connection to each other and to Canada, advance awareness of and appreciation for Canadians’ shared values, cultural diversity, symbols, and institutions, and provide opportunities for Canadians to engage in public commemorations and celebrations of Canadian events and accomplishments.
The program demonstrated support for cultural diversity and equity-deserving groupsFootnote 1 through its events and activities as well as through internal departmental policies and practices. For example, there was evidence of volunteer recruitment from a diversity of ethnocultural communities.
Although the mandate and the objectives of the transfer are clearly articulated in multiple official documents, some internal and external stakeholders raised questions related to the program’s mandate to promote the Capital.
Organizational culture
There is evidence that the integration of the CE Program within PCH and MECCE has been mostly effective and that early challenges have stabilized over time. There was successful continued delivery of events and projects throughout and after the transfer, which was seen by key informants as evidence of successful integration. There was evidence of plans to further address integration of the CE Program within PCH and specifically within MECCE.
Structure
The CE Program is well coordinated and has led to collaborations with other MECCE programs, particularly related to event delivery. The CE Program has many partnering arrangements and collaborations at the project level, both internal and external to PCH, which has facilitated program delivery. While the program’s design and delivery model offers features that support efficiency, the evaluation identified some opportunities for improvement including through the clarification of some roles and responsibilities, and by ensuring a common approach for strategic communications and collaborations.
Systems
The evaluation determined that one of the unintended impacts of the transfer of the CE Program to a federal department was the impact on the immediate efficiency of program delivery with the longer timeline required for federal contracting processes. Although major events were successfully delivered, the transfer may have led to more last-minute work in the short term. The transfer also affected the extent to which sponsorships could be used for program activities. The increased demands of the 150th celebrations negatively impacted sponsors’ commitment for subsequent years.
Recommendation
The evaluation recommends that the assistant deputy minister for Sport, Major Events and Commemorations Sector further develop, communicate and implement strategic priorities for the program, which will guide and define the program’s development and delivery.
1. Introduction
This report presents the findings from the evaluation of PCH’s CE Program. The evaluation was carried out as indicated in the PCH Evaluation Plan, 2018-19 to 2022-23. It was conducted in accordance with the Treasury Board Policy on Results (2016) and Directive on Results (2016).
This was the first evaluation of the program and covers a four-and-a-half-year period from September 30, 2013 to March 31, 2018. Given the transfer from the NCC to PCH in 2013 and with input from senior management during scoping interviews, the evaluation focused on key organizational and change management issues that might have affected program performance.
2. Program profile
Unlike the majority of PCH programs, the CE Program does not distribute funding through grants and contributions (Gs&Cs). Rather, the program organizes and delivers activities using operations and materials (O&M) funding in the following areas: major events and celebrations, commemorations, public art, interpretation and promotion of sites and symbols of national significance, representation of the provinces and territories in CCR, youth activities, and visitor services.
2.1. Program history
On September 30, 2013, as a result of Budget 2013 and subsequent amendments to the National Capital Act and the Department of Canadian Heritage Act, specific duties and functions of the NCC were transferred to PCH. According to a Memorandum of Understanding (MoU) between the NCC and PCH, the purpose of the transfer was to “promote Canada’s Capital Region through its Capital Experience” Program to ensure that a broad, national perspective be brought to celebrations and that events draw on the cultural and social fabric of the whole of Canada.
2.2. Program activities, objectives, expected outcomes, and results
2.2.1. Objectives and Expected Outcomes
The objective of the CE Program is to foster feelings of pride and belonging among Canadians. The CE Program objective and its expected outcomes are shown in Table 1. The Logic Model that links program activities to the expected outcomes can be found in Annex A.
Objective | Immediate outcomes | Intermediate outcomes |
---|---|---|
To foster feelings of pride and belonging among Canadians towards their Capital, while increasing their overall awareness of CCR as a destination where people can experience Canada’s heritage, culture, and achievements. |
|
|
Following its transfer to PCH, the CE Program appeared in the Department’s Program Alignment Architecture (PAA) as a sub-program under the program “Attachment to Canada” and supported the Department’s second strategic outcome, “Canadians share, express, and appreciate their Canadian identity.” With the approval of the Departmental Results Framework (DRF) in 2017-18, the CE Program fell under Core Responsibility 2 – Heritage and Celebration and now supports the ultimate result “Canadians are engaged in celebrations and commemorations of national significance”.
2.2.2. Activities
The core program activities that the CE Program carries out are related to the development, promotion and production of events, monuments, public art and services in CCR. Major events include Winterlude, Canada Day celebrations, and Christmas Lights Across Canada. The program also supports public art, commemorations, monuments and visitor services in CCR. The program does not deliver grants and contributions.
During the period of this evaluation, and in preparation for the extensive Canada 150 celebrations, CE Program worked with and supported the functions of the Canada 150 Federal Secretariat and participated in a Special Project Team from 2015 to 2018.
Events and activities are currently measured via physical and virtual attendance numbers, and the number of media mentions. For major events such as the 150th celebrations, regional economic impacts are also examined as appropriate.
2.3. Program management and governance
Accountability for the CE Program is with the assistant deputy minister of PCH’s Sport, Major Events and Commemorations Sector. The director general of the MECCE Branch oversees program delivery, which is undertaken jointly by two directorates: Capital Celebrations and Program Operations (CCPO); and Capital Interpretation and Commemorations (CIC). Annex C presents more details on the organizational structure.
CCPO is responsible for:
- the development of programming and the production of events and activities in CCR such as Winterlude, Canada Day celebrations, and Christmas Lights Across Canada;
- managing volunteer services, production services, and programming;
- broadcasting and new media; and
- establishing partnering agreements, contracts, other agreements, and sponsor activations through the CCPO sponsorship and business development team.
CIC is responsible for managing:
- the development and delivery of new permanent national commemorations (for example, War of 1812 Monument, National Holocaust Monument), working closely with the NCC who acts as the project implementation partner, other government departments such as Public Services and Procurement Canada, and a variety of non-governmental stakeholders;
- the delivery of temporary public art projects (for example, Byward Market Courtyards, Indigenous art installations in the Portage Bridge pedestrian tunnels) that showcase Canadian artistic excellence and cultural vitality, often working with other government departments;
- the delivery of visitor services at the Capital Information Kiosk on Parliament Hill, and at nationally significant locations such as the Centennial Flame and the National War Memorial; services include the delivery of public and school tours and programs, as well as school travel itinerary services; and
- the development of large-scale interpretation projects including the Sound & Light Show on Parliament Hill, permanent exhibitions installed on prominent sites within CCR, and smaller scale interpretation projects such as the Confederation Boulevard Banner program and Chateau Terrace photo exhibits.
The NCC is a key partner of the CE Program. A joint MoU signed by the deputy minister of PCH and the chief executive officer of the NCC in October 2013 governs the relationship between the two partners. The MoU was intended to ensure that the ongoing activities of the CE Program were maintained without interruption as of, and following, the date on which the CE Program mandate was transferred to PCH. In the MoU, both entities committed to ensuring the ongoing success of the CE Program in its commemoration of the milestones that have helped to shape Canada, as a country, as well as the implementation of a broad, national perspective in major events in CCR.
According to the MoU, PCH is the overall project manager for the CE Program, while the NCC collaborates, supports, advises, and provides required approvals in certain areas such as site approvals for events, and design and site selection for monuments. NCC also manages and maintains NCC lands and structures on these lands, including monuments and pre-existing permanent public art installations.
In 2019, the responsibility for the NCC was transferred to the Minister of Public Services and Procurement Canada.
2.4. Program resources
On September 30, 2013, assets and liabilities related to the CE Program were transferred from the NCC to PCH (including assets with an initial book value of $4,698,000). CE Program assets consisted mostly of production services, information technology, and communications equipment used for events and activities. In addition, CCPO provided overall management of inventory for festival equipment and hardware that could be rented through its Rent and Loan Program. Assets such as existing monuments, public artworks and interpretive structures were not transferred from the NCC.
As outlined in Table 2, over the period of this evaluation, actual spending on the CE program was approximately $77.6 million including salaries, goods and services, and O&M.Footnote 2 Lower expenditures in 2013-14 reflect that the CE Program was transferred to PCH mid-way through the fiscal year. PCH’s involvement in the 2017 Canada 150 events accounts for higher expenditures in that fiscal year and in the previous year leading to 2017. This funding was a one-time investment for Canada 150.
Resource | 2013-14 | 2014-15 | 2015-16 | 2016-17 | 2017-18 | Total ($) |
---|---|---|---|---|---|---|
Salary ($) | 2,501,015 | 4,998,238 | 5,428,257 | 5,452,556 | 7,481,199 | 25,861,265 |
Goods & services ($) | 3,283,233 | 10,491,927 | 9,027,291 | 11,085,549 | 16,017,157 | 49,905,157 |
O&M ($) | 177,676 | 337,287 | 354,768 | 429,235 | 488,282 | 1,787,248 |
Total ($) | 5,961,924 | 15,827,452 | 14,810,316 | 16,967,340 | 23,986,638 | 77,553,670 |
Source: PCH financial data
3. Approach and methodology
The evaluation was led by PCH’s Evaluation Services Directorate (ESD), with consultant support for the media content analysis and the development of the report. This section outlines the evaluation approach and methodology including scope, timelines, calibration, evaluation questions, data collection methods, limitations, and mitigation strategies.
3.1. Scope, timeline and quality control
The evaluation covered the period from September 30, 2013, to March 31, 2018. Based on input from program management as well as timelines and resources available, the evaluation focused on the CE Program mandate and operational challenges following its transfer from the NCC to PCH. While the effectiveness and efficiency of the programming was not an explicit focus of the evaluation, the examination of operational challenges led to addressing these issues.
The evaluation used an organizational assessment model, the Burke-Litwin Organizational Performance and Change Model, as a framework to guide the evaluation, including the development of evaluation questions, data collection, analysis of findings, and reporting. The Burke-Litwin model describes the linkages among the key factors that affect organizational performance and determine how change occurs within an organization.
While the Burke-Litwin model was generally relevant to the CE Program, the context of organizational change management in the public sector required some adaptation. The factors that were examined for this evaluation were external environment, mission and strategy, organization culture, structure, and systems. Two additional dimensions of the Burke-Litwin model, tasks and individual skills and employees’ needs and values, were initially included in the evaluation scope. However, due to the time gap between the beginning of the project and its completion, any initial challenges with the transfer had been resolved, and there were no substantial findings from these two dimensions of the model to include in the final report. Additional information on the Burke-Litwin Model can be found in Annex D.
The evaluation of the effectiveness of activities funded through the Canada 150 Fund was outside of the scope of the evaluation; however, it must be noted that the public interest, media attention, and government investment related to Canada 150 events were a significant context for both the 2017-18 fiscal year and the previous fiscal year and heavily influenced areas such as attendance, media views and funding levels.
3.2. Calibration
The evaluation was calibrated as follows:
- The scope of the evaluation was determined, in part, on the timelines and level of resources available for the project.
- The evaluation focused on those areas where the evaluation could best support management decision-making regarding its mandate and operations.
- To the extent possible, the evaluation made use of secondary data sources and pursued primary data collection only when it was determined insufficient information was available from existing sources and/or to validate existing data.
3.3. Evaluation questions
Table 3 outlines the specific evaluation questions by Burke-Litwin dimension that were used to guide the evaluation, including the development of data collection instruments and the analyses. Program results were also examined and are to be found in section 4.6 of this report. More details related to the indicators and data collection methods can be found in the Evaluation Framework (Annex B).
Burke-Litwin dimension | Evaluation questions |
---|---|
1. External Environment |
1.1. What external environmental challenges and opportunities exist which affect the CE Program? 1.2. How has the CE Program responded to these challenges and opportunities? |
2. Mission and strategy |
2.1. Do program representatives and relevant stakeholders have a clear understanding of the objectives and mandate of the CE Program following its transfer from the NCC to PCH? 2.2. To what extent are the CE Program mission, mandate, objectives and priorities, as currently stated, aligned with: the PCH DRF, core responsibilities, strategic outcomes and PCH and Government of Canada priorities; Sport, Major Events, and Commemorations Sector mission, mandate, objectives and priorities; and Major Events, Commemorations, and Capital Experience Branch mission, mandate objectives and priorities? 2.3. Does the CE Program support equity-deserving groups? |
3. Organizational culture | 3.1. How effectively has the CE Program integrated with PCH and MECCE? |
4. Structure |
4.1. Is the current CE Program design and delivery model the most effective and efficient for the delivery of the Program’s mandate? 4.2. Are there alternative delivery models that could be considered? |
5. Systems (policies and procedures, human resources and resource planning) |
5.1. What are the unintended impacts on program delivery of the CE Program transfer to a federal department? 5.2. To what extent do program resources (such as human and financial) enable the effective delivery of the CE Program activities and the achievement of its outcomes? |
3.4. Data collection methods
This evaluation used a mixed-method approach including a document review, a literature review, administrative data analysis, interviews with program representatives and key stakeholders, and a media content analysis. The following provides a description of each of the data collection methods.
3.4.1. Document review
The evaluation team reviewed a total of 38 Government of Canada documents and publications related to the delivery and mandate of the CE Program to collect background and contextual information needed to understand program operations. The review included existing program documentation, policies, MoUs, priority documents, corporate documents, industry reports, and relevant external documentation.
3.4.2. Administrative data review
The evaluation team undertook an analysis of the following CE Program administrative data:
- Financial data: The financial data analysis examined the reallocation of resources following the transfer of the CE Program to PCH, changes in program expenditures over time, and alignment between planned and actual spending.
- HR data: An analysis of HR data provided evidence of the number of FTEs assigned to programming as well as of HR challenges following the transfer from the NCC to PCH.
- Performance data: CE Program performance data supported the analysis of the program mandate and its alignment with PCH and government priorities, as well as provided evidence of operational challenges encountered by the CE Program.
3.4.3. Interviews with program representatives and key stakeholders
A total of 29 interviews were conducted as part of the evaluation and included individuals responsible for managing and delivering the program’s various components, as well as key CE Program partners. Respondents included six external stakeholders from other government departments, the NCC, and partner agencies and organizations; nine internal services representatives; eight CE Program management representatives; and six CE Program staff.
3.4.4. Literature review
The literature review entailed an analysis of published literature and websites, specifically examining models for delivery of similar programming among four capital cities: Washington, Canberra, Québec, and Regina. The four capitals were selected based on several factors, including availability of information, the similarity of the legal-administrative frameworks and the programs implemented, and inclusion of some Canadian provincial capitals to allow for comparisons within a Canadian context.
Additionally, the literature review included analysis of research related to organizational change management in the public sector, which helped to contextualize the impacts of the CE Program transfer from the NCC to PCH. The following summary of theories was used to identify factors that could not be expected to be addressed by PCH and the CE Program in their management of change as they were beyond their control:
- Public organizations are more sensitive to their environment
- Permanent change to adapt to the needs of all service delivery clients
- Greater vulnerability (for example, dissatisfaction, change in public policies, political directives)
- Public organizations have less control over change and its direction
- Decision-making is more political than managerial
- Imposition rather than explanation of the merits of changes implemented
- Actions are limited by a strong regulatory framework
- There are a considerable number of parties with issues and interests within public organizations
- Mutually legitimate expectations that can easily be in conflict
3.4.5. Media content analysis
An analysis of media content was undertaken to provide an understanding of key themes related to the CE Program, and included stories that were written, shared, and discussed in Canadian news media over the evaluation period. The media scan included English and French print and broadcast media sources, as well as other specialized content identified by the CE Program; an analysis, synthesis, and triangulation of the data collected; and the development of a draft and final technical report on the findings/results of the media analysis. Stories were sampled from April 1, 2013 to March 31, 2018, which included the period leading up to and following the transfer of the CE Program from the NCC to Canadian Heritage. In total, 1,467 articles in the quantitative analysis and 109 stories in the qualitative analyses were reviewed.
ProQuest was the main tool used for the media content analysis, supplemented by online searches for select media sources. Specialized journals covering monuments and art displays were also included in searches for relevant monuments and art installations and searches were also performed in le Droit to capture French language coverage from CCR. Finally, limited Google News searches were performed to supplement the other searches. The quantitative sample consisted of a total of 1,467 stories, including 1,288 English stories and 179 French stories. A qualitative sub-sample of 83 stories with substantive discussion on relevant topics was then drawn from this sample.
Further details regarding the media analysis strategy and findings can be found in Annex E.
3.5. Evaluation limits and mitigation
The following outlines the key constraints and limitations of the evaluation process.
Limits | Mitigation |
---|---|
Evaluation project delays. Due to challenges related to the COVID-19 global pandemic, several years had passed between the period covered by the evaluation and the finalization of the report. Multiple adjustments and improvements have since been implemented in the CE Program. Both the program and evaluation teams changed personnel, and there were limitations on the knowledge of the evaluation context and on corporate knowledge related to transfer. |
The evaluation team and the program maintained communication over the period to ensure the ongoing relevance of the findings and conclusions. |
Limited evaluation evidence to analyze the public art component of the CE Program. This component was rarely mentioned in the KIIs, documents, or media analysis. |
Public art is a relatively small component of the programming; the evaluation draws no specific conclusions related to public art. |
4. Findings
This section presents a summary of CE Program results over the evaluation period followed by the analysis of the program using the Burke-Litwin Organizational Performance and Change Model.
4.1. Program results
While this evaluation was designed to focus on the transfer of the CE Program from the NCC to PCH, the evaluation team analyzed existing performance measurement data to provide an overview of the immediate and intermediate results over the period.
CCPO Results
CCPO achieved its immediate result that Canadians have access to events and activities in CCR that celebrate Canadian identity and reflect Canada’s diversity and its intermediate outcome that Canadians participate in events and activities that celebrate Canadian identity and reflect Canada's diversity in the CCR.
Tables 5 and 6 describe trends in social media impressions and broadcast viewership between 2014-15 and 2017-18 for Canada Day, Winterlude and Christmas Lights Across Canada. The increasing numbers over time indicate an overall increase in access to events and activities in CCR.
Social media impressions | 2014-15 | 2015-16 | 2016-17 | 2017-18 |
---|---|---|---|---|
Canada Day | 3,423,677 | 691,858 | 1,054,758 | 5,667,407 |
Winterlude | - | 3,023,143 | 5,123,420 | 3,404,830 |
CLAC | 379,949 | - | 2,148,511 | 1,308,378 |
Source: Program data
Viewers of Broadcasts | 2014-15 | 2015-16 | 2016-17 | 2017-18 |
---|---|---|---|---|
Canada Day | 2,601,334 | - | 1,771,240 | 15,000,000Table 6 note * |
Winterlude | - | - | 1,134,000 | 4,453,000 |
CLAC (Facebook Live) | - | - | 200,000 | 160,000 |
Table 6 notes
- Table 6 note *
-
Source: Program data higher than usual attendance number was largely due to the 150th celebrations.
Annual, in person attendance estimates for these events was generally stable over 4.5 years, with the exception of higher numbers in 2017-2018, due to the 150th celebrations. Average attendances were:
- Winterlude: 600,000
- Canada Day: 350,000 (with the exception of 500,000 for Canada Day 2017 due to Canada 150 celebrations), and
- CLAC: 100,000 (with the exception of 16,000 in 2014-15 when attendance was lower as multimedia production was not included until 2015-16).Footnote 4
The results of a 2015 satisfaction survey on Canada Day in the Capital indicated that a majority of respondents (71%) agreed that the activities were representative of Canada and more than half of respondents (62%) agreed that their visit gave them a better appreciation of Canada's cultural and social diversity. In 2017, a poll conducted with funding from Canada 150 demonstrated that nearly two thirds of respondents (63%) agreed that the festivities were representative of Canada.
Economic Impacts
There is evidence that the CE programming contributes to positive economic impacts for the CCR. A satisfaction survey of Canada Day 2017 in the Capital was carried out by a third party on behalf of PCH and financed with funds from Canada 150. The results of the survey underscored the economic impact of this event, particularly for tourism. According to the survey, Canada Day influenced the decision of visitors to come to CCR by 90% in July 2017, which was a significant increase from 69% in 2015. The majority of respondents (88%) stayed at least one night in the region, mostly with friends or family (52%) and in hotels (29%). Visitors to the area spent an average of $622 per person.
Moreover, a recent study commissioned by the Ottawa Festival Network found that visitors (excluding residents) attending annual festivals, special events, and fairs including Winterlude, Canada Day and CLAC, spent $122 million in 2016 during their stay in the city. While economic benefits are not an intended outcome of the CE Program, these results highlight the indirect impact of the program on, for example, the tourism sector in the CCR.
CIC Results
The CIC programming met its immediate expected result that Canadians have access to information and orientation to help them experience the sites and symbols of CCR. The number of requests for information from Visitor Services increased every year between 2013-14 (515,214) and 2017-18 (865,982). This includes interactions at the Capital Information Kiosk, with information officers along Confederation Boulevard, and through Discover the Hill and War Memorial Programs. It also includes e-mail and phone requests received through the Contact Centre.
The programming also met its expected intermediate result of CIC that Canadians experience CCR and its symbols and sites. The evaluation found that the number of youth that participated in tours in CCR did not change significantly, with an average of 26,701 tour participants annually. The Sound & Light Show on Parliament Hill was completely revamped and the new show, Northern Lights, was launched in the summer of 2015. Over the summer months the show, which makes use of state-of-the-art technology, sound, and lighting to tell Canada’s story, attracted nearly 270,000 people.
Monuments and public art are continually accessible and there is no current mechanism in place to track the number of visitors.
As well, Capital Experience received numerous external awards for their work over the period covered by the evaluation:
- The Canadian Building Trades Monument and Pindigen Park both received an Ottawa Urban Design award for example, while the National Holocaust Monument received multiple international awards.
- The Sound and Light Show received an award from the International Festivals and Events Association (IFEA), which recognizes outstanding examples of quality and creativity.Footnote 5
- Multiple CCPO events received IFEA awards in this period, as well as Festival and Event Ontario awards and Ottawa Festival awards for Winterlude, Canada Day programming and CLAC.
4.2. External Environment
4.2.1. External environmental challenges and opportunities
Evaluation question: What external environmental challenges and opportunities exist which affect the CE Program?
Various challenges and opportunities originating from the external environment have had an impact on the program, including strengthened security on Parliament Hill, involvement in the delivery and logistics related to the Canada 150 celebrations, unpredictable weather conditions, and changes to the NCC governance structure. Media coverage, whether positive or negative, can impact the program delivery.
CE Program experienced some external environmental challenges
Heightened security measures were implemented following a significant public security threat on Parliament Hill in October 2014. These new measures increased overall expenditures for the CE Program and affected visitor experience. According to the results of Canada Day public satisfaction surveys, there was a decrease in satisfaction with celebrations in 2017 compared to 2015. However, CCPO increased its security protocols while improving visitor experience based on lessons learned following enhanced security requirements and crowd control for their large-scale events.
Involvement of CE Program employees in the delivery of the Canada 150 celebrations affected their workload in 2017 across most programs: the Celebrate Canada period (June 21-July 1st), Winterlude, and the Sound and Light show. The number of interactions at the Capital Information Kiosk and through the various interpretation programs also increased significantly in 2017.
Weather conditions, particularly when these were suddenly unfavourable, negatively impacted uptake of CE Program events and activities as well as visitors’ satisfaction levels. In the context of climate change, variations in climate and local weather patterns are expected to have an ongoing impact on the CE Program.
Media coverage can impact program delivery including by influencing public perceptions and participation in events. According to the media content analysis, media coverage of events in CCR increased from 262 articles in 2013-14 to 365 articles in 2017-18. This would indicate increased media scrutiny and public opinion with respect to the delivery of program activities, though most media coverage (75%) was positive. Table 7 provides an overview of the sub-sample of stories identified by the media content analysis as reporting sentiments regarding CE Program activities, categorized by event or activity type.
Stories by sentiment about activities and events, search topic, and regions | Positive (number) | Negative (number) |
---|---|---|
Events – Winterlude | 17 | 3 |
Events – Canada Day | 11 | 2 |
Events – Christmas Lights Across Canada | 11 | 3 |
Art Installations | 8 | 0 |
Monuments | 8 | 9 |
Interpretation | 4 | 1 |
Transition from NCC to Canadian Heritage | 3 | 3 |
Totals | 62 (75%) | 21 (25%) |
Source: Heritage CEP Media content analysis Technical Report
4.2.2. Program response to challenges from the external environment
Evaluation question: How has the Program responded to these challenges and opportunities?
The program implemented communications and risk management strategies to mitigate the external challenges. Specific steps taken included, but were not limited to, developing communications strategies for managing the impact of adverse weather conditions and providing media training for employees to better navigate public opinion of events and activities. The program worked with security partners, including the Parliamentary Protective Service for its events on Parliament Hill, to develop and put in place security plans to ensure the safety and security of event attendees. However, there may be opportunities for the CE Program to improve further its collaboration and communication with external stakeholders on operational and programming matters.
The CE program undertook several strategies to manage the effects of external environmental factors
KIIs and CE Program documents indicated that the CE Program implemented effective risk mitigation strategies to manage the effects of factors from the external environment. The most important mitigations were related to public security and its impact on planning, financial resources, and crowd management. Specifically, the program developed and implemented specific risk management strategies for each major event, such as Winterlude and Canada Day. KIIs reported that the program managed the safety and security of events in close collaboration with the program’s many security partners.
The program is aware of the need to address the effects of climate change, which impacts participation and experience. The document review noted the existence of contingency plans for program delivery, with solutions such as changing delivery dates or the type and location of events. Key informants also noted strategies were in place to manage the effects of changing and adverse weather conditions. These included a risk management strategy with contingency plans to deal with certain unforeseen events related to the weather, such as extreme heat during the summer months, as well as media training offered to staff to assist with dealing with public inquiries.
There are opportunities for the CE Program to further address external environmental factors
According to key informants and review of program documents, the program could continue and further address external factors through strengthened partnerships and additional risk management. KIIs identified the positive results of the department’s more recent participation in the Ottawa and Gatineau tourism committees. Other partnership opportunities include further development of the community partner-based approach taken for the delivery of Winterlude 2019.
At the time of the transfer, the NCC reported to the Minister of Canadian Heritage. There was some suggestion that this reporting relationship created a perceived conflict of interest for PCH participation in committees with external stakeholders, including the NCC. However, in 2016, the NCC changed their governance structure, and in 2019 was transferred out of the PCH portfolio to Public Services and Procurement Canada. This represented new opportunities for the program to engage with the NCC and other external partners, such as the mayors of Ottawa and Gatineau, to develop strategic relationships and collaborations.
4.3. Mission and strategy
4.3.1. Mission and strategy: understanding of program
Evaluation question: Do program representatives and relevant stakeholders have a clear understanding of the objectives and mandate of the CE program?
Although the mandate and the objectives of the transfer are clearly articulated in multiple official documents, internal and external stakeholder understanding was divided. Half of those interviewed indicated that the mandate of CE that focuses on the promotion of the Capital is unclear. Nonetheless, there was some improvement in MECCE employee satisfaction scores on whether the sector clearly expresses its vision, mission, and objectives, as noted in the PCH Employee Survey.
The objective of the CE program transfer and its mandate were consistent in documentation
Several program documents outlined the objectives and mandate of the CE Program post-transfer. For example, the 2013 Economic Action Plan Act stated that the purpose of the transfer was to ensure that a perspective was brought to celebrations and that events draw on the cultural and social fabric of the whole of Canada.Footnote 6 Additionally, the MoU developed between the NCC and PCH when the program was transferred stated that its objective was to ensure that “ongoing activities of the CE Program are maintained without interruption as of the September 30, 2013 transfer.”Footnote 7
The Department of Canadian Heritage Act was amended in 2013 to include the CE Program. The Act has the following paragraph added, under the description of the Minister’s powers, duties and functions: (k.1) the organization, sponsorship and promotion of public activities and events, in the National Capital Region as defined in section 2 of the National Capital Act, that will enrich the cultural and social fabric of Canada, taking into account the federal character of Canada, the equality of status of the official languages of Canada and the heritage of the people of Canada.
Key informants provided mixed feedback on mandate clarity
Program representatives and stakeholders interviewed were divided on whether the mandate of the CE Program was clear following the transfer. While about half of the CE Program staff and external partners interviewed indicated that the mandate was clearly defined, there were also questions on whether promotion of the Capital remained an objective now that the CE Program was in a department with a national mandate.
4.3.2. Mission and strategy: alignment with program mission, mandate, objectives and priorities
Evaluation question: To what extent are the CE program mission, mandate, objectives and priorities, as currently stated, aligned with:
- the PCH DRF, core responsibilities, strategic outcomes and PCH and Government of Canada priorities;
- SMEC mission, mandate, objectives and priorities; and
- MECCE mission, mandate objectives and priorities?
The objectives and mandate of the CE program, including its two key components, clearly align with the mandate and priorities of PCH and the Government of Canada, MECCE, and SMEC. The program supports PCH’s strategic outcomes related to Canadians’ engagement in celebrations and commemorations of national significance, and to ministerial priorities of diversity and inclusion. The program is well aligned with MECCE’s mandate to strengthen Canadians’ sense of connection to each other and to Canada. Further, CE Program contributions were well articulated in PCH performance reports. The CE Program thus benefited from good visibility in relation to the alignment of its activities with government priorities, particularly with respect to the sharing, expression, and appreciation of the Canadian identity.
Program mandate, objectives, and priorities aligned with PCH DRF, PCH core responsibilities and strategic outcomes, and PCH and Government of Canada priorities
The objective of the CE Program is to foster feelings of pride and belonging among Canadians towards Canada’s Capital, while increasing overall awareness of CCR as a destination where people can experience Canada’s heritage, culture, and achievements. The CE Program aims to ensure that Canadian values, stories, and symbols are represented in its activities and events.
In 2017-18, the CE Program was integrated into the DRF under Core Responsibility 2 – Heritage and Celebration and its objective supported the result “Canadians are engaged in celebrations and commemorations of national significance.” Prior to the creation of the DRF, the CE Program was also well aligned with the Department’s strategic outcome, “Canadians share, express, and appreciate their Canadian identity”, which was previously part of the Program Activity Architecture (PAA).
It was aligned with the government priority of diversity - highlighted in the 2015 Speech from the Throne – as the program aimed to showcase Canada’s diversity throughout its celebrations, interpretation, commemorations and public art.Footnote 8 Further, the program is currently exploring how and to what extent it can contribute to Sustainable Development Goals (SDG).
Program mandate, objectives, and priorities aligned with mission, mandate, and objectives of SMEC and MECCE
The CE Program is well aligned with the mission, mandates, objectives and priorities of the sector and branch charged with its delivery, SMEC and MECCE. It is aligned with the MECCE branch mission of promoting active participation, recognizing the common values of Canadians and celebrating the diversity of people and events that shape Canada. MECCE plays a leading role in carrying out interpretation, commemoration and awareness-raising activities and programs aimed at the general public, the promotion of sites and symbols of national importance, as well as efforts to promote the Capital as a place of pride and discovery for all Canadians.
The CE Program objective also supported the overall mandate of MECCE to strengthen Canadians’ sense of connection to each other and to Canada, advance awareness of and appreciation for Canadians’ shared values, cultural diversity, symbols, and institutions, and provide opportunities for Canadians to engage in public commemorations and celebrations of Canadian events and accomplishments.
4.3.3. Mission and strategy: support for Equity-Deserving Groups
Evaluation question: Does the CE Program support equity-deserving groups?
The CE Program demonstrated support for cultural diversity, as well as government and PCH priorities for equity-deserving groups. There was evidence that internal department policies and practices include consideration of diverse and inclusive hiring practices. The activities and events delivered by the programming highlighted culturally and linguistically diverse artists and paid particular attention to Indigenous culture.
Internal policies and procedures supported diversity
Analysis of CE Program performance data demonstrated that existing policies and procedures reflect Canada’s diversity. For example, 25% of the volunteers involved in CE Program events and activities were born outside of Canada; this compares to the 21.9% of Canada’s population that is born outside of Canada according to the 2016 Census.
CE Program events and activities promoted diversity
The majority of KIIs agreed that diversity and inclusion were well integrated into the program. KIIs emphasized the strong involvement of artists from diverse groups, including Indigenous and racialized peoples and LGBTQ2+, in the delivery of activities, particularly during Canada Day and Winterlude. For example, the Canada Day report (2017) noted that 700 artists were invited and that the program made significant efforts to reflect Canada's cultural diversity by involving artists from various linguistic and cultural backgrounds. Some interviewees also emphasized that Canada Day had increased emphasis on regional diversity, Indigenous cultures, and multiculturalism. Other KIIs cited Winterlude as a key contributor to diversity efforts, as it consistently incorporated Inuit games among its activities, as one example.
Documents show CIC and partners mounted temporary public exhibits featuring Anglophone and Francophone artists, and, in 2016 and 2017, specifically highlighted Indigenous and female artists.
The media analysis provided evidence of how programming highlighted cultural diversity. Canada’s cultural diversity was addressed in 31% of stories analyzed, and related to monuments, Canada Day, and CLAC. Monument stories, including those related to the Holocaust or escapes from communism, often referred to multiple generations of Canadians of various origins and the struggles faced by immigrants. There was also recognition of Indigenous peoples, for example with the erection of a monument to 17th century Algonquin leader Chief Tessouat, considered to be “very symbolic and very important to the Anishinabeg community.”Footnote 9
4.4. Organizational culture
4.4.1. Organizational culture: CE integration with PCH and MECCE
Evaluation question: How effectively has the CE Program integrated with PCH and MECCE?
There is evidence that the integration of the CE Program within PCH and MECCE has been mostly effective. There was successful continued delivery of events and projects throughout and after the transfer. Though there were initial challenges with the integration, KIIs noted that the situation had stabilized over time.
The integration of the program was mostly successful
The successful continued delivery of events and activities is evidence of successful integration of the programming within PCH. Employee feedback has also been mostly positive.
The results of the 2017 PSES demonstrated that the majority of CE Program employees had resettled into new working circumstances following the transfer of the CE Program from the NCC to PCH (more in section 4.7). Several KIIs confirmed that integration was generally successful despite some challenges experienced during the initial transition period.
The results of the document review found some examples of actions undertaken to facilitate the integration of the CE Program into PCH. For example, the Human Resources Plan (2015) of the MECCE Amalgamation of Branches stated that that process could be an opportunity to address outstanding issues, and update and evaluate staff positions in their new organizational structure.
4.5. Structure
4.5.1. Structure: Effectiveness and efficiency of current design and delivery model
Evaluation question: Is the current CE Program design and delivery model the most effective and efficient for the delivery of the Program’s mandate?
While the program’s design and delivery model offers features that support efficiency, the evaluation identified some opportunities for improvement. The manner in which CE Program delivery is organized inside MECCE has led to collaborations among programs, mostly with respect to event delivery. Consistent with its outward-facing role, the CE Program has many partnering arrangements and collaborations at the project level, both internal and external to PCH, which has facilitated program delivery. Additional efficiencies could be found through clarification of some roles and responsibilities and to ensure a common approach for strategic communications and collaborations.
The integration of the CE Program model into PCH was generally complete at the time of the evaluation, but some challenges remained in terms of adapting the business-oriented model of the NCC to the realities of a federal department. Specifically, the CCPO components were based on a model with business lines that retained decisional flexibility, contracting, and sponsorship flexibility.
As a result of the difference between this model and the operational requirements of the federal government context, KIIs identified some obstacles with procurement, sponsorship, and communications. Procurement challenges included lack of role clarity, understaffing of procurement and subsequent delays in processes, and different finance and contracting procedures (for example, many levels of approval required). However, several changes, particularly communications, have since been implemented that have improved these elements of CE Program delivery.
There was complementarity between the CE Program and other MECCE activities
Although there were similarities between the CE Program and other MECCE activities, several KIIs stated that there was no obvious or intentional duplication of efforts between the two bodies. In fact, the 2017 Performance Information Profile demonstrated complementarity between the CE Program and MECCE’s State Ceremonial and Canadian Symbols program (which complements the CE Program as it manages national ceremonies, including State funerals, installations of new Governors General, and the protocol element of the Canada Day Noon Show) and the Celebration and Commemoration Program (which complements the CE Program as it funds the delivery of Canada Day outside the Capital region and commemorations of national scope through grants and contributions).
There was evidence of collaboration between the CE Program and MECCE. For example, most KIIs recognized and appreciated the Collaborations and Synergies document created in 2017 which outlined the many potential collaborations and synergies among the five MECCE units (i.e., CE Program and MECCE combined), including in the delivery of events, volunteer services, programming, sponsorship and partnership development, production services, broadcasting, and Visitor Services.
Roles and responsibilities were not clearly defined in the MoU between the NCC and PCH
The MoU governing the relationship with the NCC listed areas of continued shared responsibility between the NCC and PCH, as well as areas of distinct responsibility. It referred frequently to coordination and consultation between the NCC and PCH with regards to the CE Program, with 19 appendices to govern this relationship. PCH was the overall project manager for the CE Program, while the NCC supported PCH in certain areas (for example, design and site selection) and managed and maintained NCC lands and associated structures.
The MoU stated that NCC and PCH would each manage their own volunteers, sponsorships, interpretation activities, and tours but responsibilities were divided for some programs and activities (for example, Flag/Banner Program, National Celebrations on NCC land). CIC retained most of the responsibility for programs that were managed jointly by PCH and NCC, such as commemorations. Where projects had joint responsibilities, PCH retained overall project management responsibilities. PCH was also responsible for policy, planning and design, while the NCC was generally responsible for construction, maintenance, and ongoing costs.
However, the evaluation identifies some gaps in understanding of roles and in communication. No revision date was set in the MoU, though it stated that both parties agreed to “jointly develop annual plans for priority projects with a three-year outlook.” No such plans were available at the time of the evaluation and there was no evidence that a review exercise was taking place. Some key informants suggested needs to clarify and further detail operational processes and lines of communication between PCH and NCC, which was transferred to the PSPC portfolio in November 2019. Finally, staff turnover and the subsequent loss of corporate knowledge amplified uncertainties and further underscored the need for enhanced clarity regarding the respective roles and responsibilities of PCH and the NCC.
Existing collaborations were identified but there are further opportunities to enhance synergies within PCH, between PCH and the NCC, and between PCH and other external partners
The majority of KIIs identified the existence of formal and informal communication and collaboration mechanisms with, for example, various tourism committees, the cities of Ottawa and Gatineau, and the Library of Parliament. PCH and the NCC collaborated for activity and event delivery, particularly for one-time projects such as monuments, which each had a working committee co-chaired by the sponsoring stakeholder and Assistant Deputy Minister (ADM). However, most KIIs reported that the relationship between the NCC and PCH would benefit from more regular and strategic communication efforts, and a few KIIs suggested the creation of a committee of all federal government agencies engaged in similar efforts in CCR. To support these efforts, several KIIs suggested the idea of negotiating a new protocol agreement that could include having a senior official designated to maintain communication with their counterpart at the NCC.
4.5.2. Structure: Alternative delivery models
Evaluation question: Are there alternative delivery models that could be considered?
Based on a review of models used in other major cities, there are alternative delivery elements that could be considered for the delivery of programming. Some features worthy of further exploration include major event delivery by multiple agencies and the use of external funding.
Four alternative delivery models offer considerations for program delivery
From an examination of the delivery models used by similar programming offered in four capital cities, Washington, D.C., Canberra, Australia, Quebec City, Quebec, and Regina, Saskatchewan, the evaluation identified a number of promising practices. All four capitals maintained advisory committees of experts on commemorative themes and public art. There are formal expert committees for the validation and approval of commemorative themes, monuments, and public art in Canberra and Washington, D.C. (inscribed by law), and Quebec City. This was also part of the NCC model for programming but has not been carried over to the CE model. Since the transfer of CE, there is no longer a formal advisory committee and advice has been sought on an ad hoc basis.
With respect to finances, some jurisdictions rely on external funding to facilitate event and activity delivery. For example, the EventACT agency in Canberra must contribute to its own income, including through ticketing and licenses.
There was no strong support among KIIs for the implementation of a new delivery model for the programming, although some noted value in exploring revenue diversification. Some of the KIIs reported that alternative delivery models would not necessarily improve the effectiveness or efficiency of the CE Program. For example, they did not foresee any benefits to integrating the private sector into program delivery, and a few indicated that privatization could result in loss of emphasis on government priority messages.
4.6. Systems
4.6.1. Systems: Unintended impacts on program delivery
Evaluation question: What are the unintended impacts on delivery of the CE Program transfer to a federal department?
Major events were successfully delivered; however, the transfer may have led to more last-minute work, and additional workflow implications due to the policy and regulatory differences between a Crown corporation and a federal department. These affected the extent to which sponsorships could be used for events, and the turnaround times for the hiring of artists for major events.
KIIs reported more delays in approvals and contracting due to federal environment
KIIs reported that delays in approvals, more complex departmental planning and budgeting cycles, and stricter rules regarding contracting and hiring practices may have had the unexpected effects of creating more last-minute work and increased overtime. Specifically, delays in planning caused by more extensive and rigid procedures for approvals meant that tasks were often performed at the last minute, particularly in procurement.
The transition of the CE Program from a Crown corporation to a department had an impact on staff and workflow. Several KIIs noted that there were additional levels of approval to navigate for decision-making following the transfer, which prolonged processes and reduced efficiency. They also reported that the CE Program faced restrictions in relation to the extent to which sponsorships could be used for events, which was a significant financial obstacle.
Challenges related to sponsorship are further detailed in the following section (4.6.2).
4.6.2. Systems: Impact of Program resources on achievement of outcomes
Evaluation question: To what extent do program resources (such as human and financial) enable the effective delivery of the CE Program activities and the achievement of its outcomes?
The transfer of the CE Program from the NCC to PCH had impacts on the delivery and results of the programming, particularly in the beginning. Transferred HR resources (FTEs) were not fully available to CE for program delivery. Eighty-one FTEs were transferred from the NCC to PCH on September 30, 2013. Just under two thirds of indeterminate and term FTEs were assigned to CCPO or CIC, while one third were transferred to other areas within PCH, such as communications and finance. Sponsorship faced funding gaps and barriers related to the misalignment of the federal sponsorship model to the former model at the NCC, however by 2017-18, revenue had met goals. While CE events and projects were generally delivered as planned, these HR resource factors had implications for the efficiency of CE Program delivery.
Impact of the allocation of HR resources upon transfer to PCH
Approximately 81 FTEs were transferred from the NCC to PCH on September 30, 2013. According to PCH human resource data, the CE Program received 52 indeterminate FTEs as well as 4 non-indeterminate FTEs with the transfer. The program also had 13 part-time students at the time of transfer. Six other units received a total of 24 indeterminate FTEs from the NCC transfer, with the largest proportion, 11 FTEs, going to corporate communications.
KIIs stated that they felt that they were not receiving support from Internal Services equivalent to the 24 FTEs transferred to those areas, as those FTEs were assigned to other work within PCH, rather than being specifically assigned to CE. However, managers reported improved services towards the end of the evaluation period.
Sponsorship met its goals despite initial barriers
The NCC business model for major events depended on revenues and in-kind contributions raised through corporate sponsorship. At the time of the transfer, the TBS transferred $2M from PCH’s annual appropriations into a vote-netted account to finance and benefit from corporate sponsorships. CE Program revenues from sponsorship and other sources surpassed the goal of $1.16M for sponsorships in 2015-16 (final surplus of $414,000) and 2017-18 (final surplus of $86,000).
Although the 2017 celebrations saw a significant increase in sponsorship, with the concomitant revenue increase, years following the evaluation period have offered challenges. Sponsorship in the context of a federal department can create complexity for CCPO and PCH Communications. Further, large investments made for 2017 meant businesses often reduced their sponsorship allotments for subsequent years. Finally, the virtual events necessitated by the pandemic increased sponsorship challenges in recent years.
5. Conclusions
The Burke-Litwin model enabled the examination of factors related to organizational change, and the impact of the transfer of the program from the NCC to PCH. Overall, the findings highlight the importance of change management best practices including strong internal and external communications and of strengthened partnerships.
Although not the focus of the evaluation, there is evidence that the programming achieves or contributes to expected results through the delivery of events, programming initiatives and visitor services. The programming increases access to events and activities in CCR and nationally via media, that celebrate Canadian identity and reflect Canada’s diversity. It delivers services that improve the availability of information and orientation that helps visitors experience the sites and symbols of CCR.
The evaluation demonstrated that the mandate and objectives of the CE Program and its two components (i.e., CCPO and CIC) are clearly aligned with the PCH DRF and the main responsibilities, mandate, and objectives of the MECCE branch. It was also well aligned with federal government priorities, particularly the promotion of diversity through its celebrations, commemorations, and public art.
The purpose of the CE Program’s transfer to PCH was to ensure that, moving forward, a “broad, national perspective is brought to celebrations, and that events such as Winterlude and Canada Day represent the cultural and social fabric of the whole of Canada.” While the objective and mandate of the transfer was articulated and communicated in multiple official documents, half of KIIs interviewed indicated that the mandate was not clear, specifically whether the mandate to promote the Capital remained.
The integration of the CE Program model into PCH was generally complete at the time of the evaluation, but there remained some challenges in terms of adapting the CE Program model to the realities of a federal department. This was particularly evident for procurement and communications processes, which entailed greater flexibility at the NCC than at PCH. However, strategies implemented since the completion of the evaluation resolved some of these concerns.
Specifically, it should be noted that between the end date of the evaluation period and the publication of this evaluation report, the relationship between the CE Program and the PCH Communications team has continued to evolve and strengthen. For CCPO, a new approach was established in 2019 where Communications now has a dedicated team of marketers and communication specialists assigned to CCPO to ensure continuity in services and expertise. CIC has worked closely with its communications colleagues to develop an integrated approach to promotion. Communications colleagues take part in weekly program meetings to identify opportunities for promotion and outreach, especially on key priorities, and to enhance the program’s visibility.
Significant progress has also been made since the end of the evaluation period in the areas of security and crowd management. Since 2017, CCPO has developed a crowd management expertise within its directorate, completing a thorough analysis of event security posture by consulting experts in the field of crowd management. CCPO implemented measures to reduce the public’s wait time and established a process with security partners to improve visitor experience, resulting in a significant reduction of wait time for Canada Day 2018.
There is no apparent duplication of the program offering, although similar activities are delivered by several MECCE units. The CE Program had formal and informal partnerships with the NCC related to tourism and formal partnerships for the delivery of many projects, such as monuments. There was evidence of the need for further clarification of the roles and responsibilities in the MoU, particularly the operational processes between the NCC and PCH. The majority of KIIs believed that the relationship between the NCC and PCH would benefit from more regular and strategic communications, and that the program could be improved by enhancing relationships with other external partners (for example, the cities of Ottawa and Gatineau).
6. Recommendations, management response and action plan
The evaluation recognizes that many of the challenges related to the transfer of the CE program from the NCC to PCH have been addressed since this project was launched. The program has done considerable work since the period of the evaluation to support the integration of CE into the department and the continued achievement of results. Therefore, and given the findings and conclusions, the evaluation identifies one broad area of recommendation: to further develop and communicate and implement strategic priorities to guide and define the program’s development and delivery moving forward.
Recommendation 1
The evaluation recommends that the assistant deputy minister for Sport, Major Events and Commemorations Sector further develop, communicate and implement strategic priorities for the program, which will guide and define the program’s development and delivery.
Management response
The Major Events, Commemorations and Capital Experience Branch (MECCE) accepts this recommendation, acknowledges the report findings, and agrees that the areas related to the transfer have been largely addressed. While the transfer of Capital Experience (CE) from the National Capital Commission (NCC) presented challenges, much has been resolved and additional progress has been made since the evaluation period ended. In addition to the areas mentioned in the report, progress has been made in the following specific areas since 2017-18:
National Vision: While CE has consistently maintained a national perspective in its programming, the pandemic pushed the program to expand the national reach of events. Canada Day 2020 offered, for the first time ever, a completely virtual Canada Day celebration which included two national broadcasts with programming from cities across the country, as well as enhanced online content. The same national virtual reach occurred for Winterlude in 2021. National planning and delivery were strengthened by working with a broad interdepartmental framework via the Interdepartmental Commemoration Committee, using whole-of-government anniversaries to guide new segments for the Sound and Light Show, temporary exhibits and thematic Confederation Boulevard banners.
Partnerships: In 2019, CE reviewed Winterlude to ensure it remained a vibrant winter festival in Canada’s Capital Region, which resulted in a more urban approach with an increase in strategic partnerships. For example, the 2019 edition of Winterlude saw a significant shift of programming and activities being offered in partnership with Business Improvement Areas (BIAs).
As the focus has clearly shifted away from the transfer of the program to Canadian Heritage from the NCC, timing is right to focus attention on developing strategic priorities to guide a successful future. The development of these priorities will help to clarify roles and responsibilities while informing strategic partnerships and effective delivery that respond to the needs of Canadians. MECCE understands that effective and timely implementation of the Action Plan is necessary to ensure sustained program improvement.
Action Plan Item | Deliverables | Timeline | Responsible |
---|---|---|---|
1.1 Hold initial planning meetings to situate CE within current MECCE activities and identify common challenges and potential synergies | 1.1.1. Initial meetings with internal stakeholders and summary meeting notes with next steps | March 31, 2022 | Director General (DG), MECCE with support from all directorates |
1.2. CE to articulate short- and medium-term priorities | 1.2.1. Approved list of priorities and strategic plan | December 31, 2022 | DG, MECCE; Directors, CE |
1.3. CE to communicate strategic priorities to employees and relevant partners | 1.3.1. DG/Directors-level meeting(s) with employees, and with stakeholders and partners to discuss priorities and articulate steps for future collaboration | March 31, 2023 | DG, MECCE; Directors, CE |
Full implementation date: March 31, 2023
Annex A: Evaluation framework
The evaluation framework below presents the evaluation questions, indicators, and data collection methods that were used for the evaluation.
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Indicators | Data sources | Methods of data collection |
---|---|---|
|
|
|
Annex B: Capital Experience Program Logic Model
Annex C: Capital Experience Program Organizational Chart
Annex D: Burke-Litwin model
A visual of the Burke-Litwin model and the dimensions of the model which were applied in the evaluation of the CE Program are described below.
Dimensions of Model | Description |
---|---|
External Environment | This includes such factors as markets, legislation, competition and the economy. All of these will have consequences for organizations. |
Mission and Strategy | An organization’s mission articulates its reason for existing. It is the foundation upon which all activity should be built. The strategy then sets out, in broad terms, how the organization will go about achieving its mission. Often, the strategy will be developed in light of environmental change and will have a significant impact on the organization’s work. |
Leadership | This considers the attitudes and behaviour of senior colleagues and how these behaviours are perceived by the organization as a whole. The way in which change is implemented and accepted through the organization will be largely influenced by the top team. |
Organizational Culture | Organizational culture can be described as “the way we do things around here”. It considers the beliefs, behaviours, values and conventions that prevail in an organization. Culture change does not happen overnight. It evolves over time as a result of many other changes in the organization. |
Structure | Very often, changes in strategy can lead to changes in the way the organization is structured. This can impact on relationships, responsibilities and ways of working. |
Management Practices | Considers how managers use human and material resources to carry out the organization’s strategy, and the style of management and their relationship to subordinates. |
Systems (policies and procedures) | Relates to an organization’s policies and procedures, including systems for reward and performance appraisal, management information, human resources and resource planning. |
Work Unit Climate | This considers employees’ perceptions of their immediate colleagues and working environment. Our immediate working environment is often what shapes our view of the organization as a whole and influences the extent to which we feel satisfied in our jobs. Changes to the immediate working environment need to be managed sensitively, as they are likely to invoke a range of emotional and political responses from staff. This is particularly the case where change involves moving location, a change in personnel, or a change in terms of conditions of service, such as working hours. |
Task Requirements and Individual Skills/Abilities | Change at a higher level in the organization will often require changes in the work carried out and the skills available in the team. |
Motivation | Considers the significance of individual and organizational goals. Motivation is key to effective change. The real challenge is to maintain motivation throughout a change project, particularly when change is often not well-received by those affected. |
Individual Needs and Values | Changes to team membership can mean a change in the team dynamic. In a perfect world, we would be able to recruit the exact fit for our teams, in terms of personal style, abilities and skills mix. However, in reality it is not always possible, and it is necessary to identify any risks in this area and mitigate them. |
Individual and Organizational Performance | Considers the level of performance in terms of productivity, customer satisfaction, quality. |
Annex E: Media Analysis Strategy
1.0 Media Analysis Overview and scope
The analysis included French and English print and broadcast media sources on topics relating to the CE Program. These studied specific issues of the external environment in which the Program operates, including the mission and strategy of the Program, and others, as identified in the Evaluation Matrix. Specific research questions were developed based on the evaluation questions and areas where the media analysis could provide complementary evidence for the evaluation.
In order to narrow the scope to topics that would most likely to be covered in media publications, the media analysis focused on specific events, monuments, and public art installations overseen by the CE Program. In consultation with Program staff, specific recent and high-profile monuments and public art installations were identified for inclusion in the search terms and the analysis.
2.0 Sources for the media analysis
A preliminary assessment of MediaScope as a potential database source for the media analysis was conducted. However, due to the limited availability of data (i.e., only for 2017-18 and 2018-19), it was determined that an alternative database should be used for the analysis. ProQuest was therefore the main tool used for the media analysis, supplemented by online searches for select media sources. ProQuest is a global information content and technology company providing applications and products for libraries, with access to more than 3,000 of the world’s news sources.
Print and broadcast media sources from across Canada were consulted. The Canadian Newsstream database available through ProQuest was the primary source for the media analysis. This database includes 588 English and French online, print, and broadcast news sources from across Canada.Footnote 10 Specialized journals covering monuments and art displays were identified for inclusion by CE Program management and staff and will also be included in searches for relevant monuments and art installations (either through ProQuest or direct online searches):
3.0 Search criteria and sampling strategy
Articles were sampled from April 1, 2013 to March 31, 2018, which included the period leading up to and following the transfer of the CE Program from the NCC to PCH.
Search terms were developed to capture a sample of media coverage related to the CE Program transition from NCC to PCH, as well as specific relevant activities, events, monuments, and art installations. The search terms drew from a list of suggested search terms from CE Program staff and were adjusted to maximize the relevance of the search. Depending on the topic searched, one or more databases were used to ensure a more targeted approach (for example, monuments will be searched through the Canadian Newsstream and specialized journal media sources). For events, the sample was stratified by different fiscal years with the purpose of analyzing trends in media coverage over time, particularly in relation to Canada 150 which occurred in 2017-18 (pre-, during, and post-Canada 150, where applicable).
Annex F: Bibliography
- ACT government. n.d. “2025 Major Events Strategy for the ACT” (PDF format).
- — — — “ACT Event Fund 2019: Funding round guide for applicants” (PDF format).
- — — — “Act Event Fund 2019. Handy Hints” (PDF format).
- Barton, Cunningham J., et James S. Kempling. 2009. “Implementing change in public sector organizations”. Management Decision 47 (2): 330-44.
- Boyne, George A. 2006. “Strategies for Public Service Turnaround: Lessons from the Private Sector?” Administration & Society 38 (3): 365–388.
- Burke, W. Warner, and George H. Litwin. 1992. “A Causal Model of Organizational Performance and Change”. Journal of Management 18 (3): 523-45.
- Chief Minister, Treasury and Economic Development Directorate. “Annual Report 2017-2018” (PDF format).
- City of Ottawa. 2012. “Renewed Action Plan for Arts, Heritage and Culture (2013-2018)” (PDF format).
- City of Québec. n.d. “Directives pour l’organisation d’un grand événement” (in french only). Accessed November 1, 2019.
- Commemorative Works Act. n.d. Public Law 99-652; 40 U.S. Chapter 89.
- Congressional Research Service. 2015. “Commemorative Works in the District of Columbia: Background and Practice” (PDF format).
- Department of Infrastructure, Regional Development and Cities. n.d. “Annual Report 2017-2018”.
- Department of Infrastructure, Regional Development and Cities. n.d. “National Capital Authority”. Infrastructure, Regional Development and Cities. Accessed November 2, 2019.
- Evaluation Service Directorate. 2019. “Terms of Reference of Capital Experience Program”.
- Government of Australia. n.d. The Territory for the Seat of Government. Vol. 18 of 1928.
- Government of Saskatchewan. n.d. “Public Accounts 2017-2018. Volume 2. General Revenue Fund Details”.
- Government of Canada. 2013. “Jobs, Growth and Long-term Prosperity – Economic Action Plan 2013” (PDF format),
- Klarner, Patricia, Gilbert Probst, et Richard Soparnot. 2008. “Organizational Change Capacity in Public Services: The Case of the World Health Organization”. Journal of Change Management 8 (1): 57–72.
- Kuipers, Ben S., Malcolm Higgs, Walter Kickert, Lars Tummers, Jolien Grandia, et Joris Van Der Voet. 2014. “The Management of Change in Public Organizations: A Literature Review”. Public Administration 92 (1): 1-20.
- Longo, Francesco. 2015. “Managing Public Reforms Effectively: A Strategic Change Management Approach”. In Strategic Change Management in the Public Sector, 1-20. John Wiley & Sons, Ltd.
- Longo, Francesco, et Daniela Cristofoli. 2007. Strategic change management in the public sector: an EFMD European case book. Chichester, West Sussex, England; Hoboken, N.J.: John Wiley & Sons.
- Mcadam, Rodney, and Neil Mitchell. 1998. “Development of a business process re-engineering model applicable to the public sector”. Total Quality Management 9 (4-5): 160-63.
- Ministry of Central Services. n.d. "Annual Report 2017-2018".
- National Capital Act. n.d. R.S.C (1985). c. N-4.
- National Capital Authority. n.d. “Annual Report 2017-2018”.
- National Capital Authority. 2019. “National Capital Authority”. Other. November 2, 2019.
- — — — n.d. “Guidelines for Commemorative Works in the National Capital”.
- National Capital Commission Act. n.d. RLRQ c C-33.1. Accessed November 1, 2019.
- National Capital Commission. n.d. “Annual Report 2017-2018”. Accessed November 1, 2019.
- “National Capital Commission of Quebec”. n.d. National Capital Commission of Quebec. Accessed November 1, 2019.
- National Capital Commission of Quebec. n.d. “Commemoration Policy in the Capital” (PDF format) (in French only).
- National Capital Planning Commission. 2005. “Memorial Trend and Practice in Washington DC” (PDF format).
- ——— n.d. 2018. “Federal Employee Viewpoint Survey 2018” (PDF format).
- ———.n.d. “Performance and Accountability Report 2017”.
- Provincial Capital Commission. n.d. “Annual Report 2018-2019”.
- Provincial Capital Commission | Saskatchewan Boards, Commissions and Agencies. Government of Saskatchewan. Accessed November 1, 2019.
- Stevens, Quentin. 2013. “Planning Canberra’s Memorial Landscape”. Fabrications 23 (1): 57-83.
- The Provincial Capital Commission Act. Vol. Chapter P-30.011 of The Statutes of Saskatchewan, 2017 (effective June 12, 2017) as amended by the Statutes of Saskatchewan, 2019, c.18.
- Thomas, Paul G. 1996. Beyond the Buzzwords: Coping with Change in the Public Sector. International Review of Administrative Sciences 62 (1): 5-29.
- Treasury Board of Canada Secretariat. 2002. “Policy on Reporting of Federal Institutions and Corporate Interests to Treasury Board Secretariat”.
- Washington, DC, Fourth of July Celebration (U.S. National Park Service). n.d. Accessed November 2, 2019.
- “Welcome to Wascana Centre - A Place For You”. n.d. Accessed November 3, 2019.
- “What We Do (U.S. National Park Service)”. s. d. Accessed November 2, 2019.
©Her Majesty the Queen in Right of Canada, 2022
Catalogue Number: CH7-67/1-2022E-PDF
ISBN: 978-0-660-42299-2
Page details
- Date modified: