Evaluation of the Canadian Conservation Institute - 2010-11 to 2015-16

Period from 2010-11 to 2015-16
Evaluation Services Directorate
July 17, 2017

Cette publication est également disponible en français.
This publication is available in HTML and PDF format on the Internet at canada.ca./canadian-heritage
© Her Majesty the Queen in Right of Canada, represented by the Minister of Canadian Heritage, 2018
Catalogue No CH7-12/2017E-PDF
ISBN 978-0-660-20489-5

Table of contents

• Executive summary
• 1. Introduction
  • 1.1. Purpose
• 2. Program profile
  • 2.1. Background
  • 2.2. Program management, governance, target groups, key stakeholders and partners.
  • 2.3. Program resources
• 3. Evaluation methodology
  • 3.1. Scope of the evaluation, calibration and quality control
  • 3.2. Evaluation questions by issue area
  • 3.3. Evaluation methods
• 4. Findings - relevance
  • 4.1. Core issue 1: continued need for the program
  • 4.2. Core issue 2: alignment with government priorities
  • 4.3. Core issue 3: alignment with federal roles and responsibilities
• 5. Findings - performance
  • Context 2010-11 to 2015-16 and structural impacts on CCI
  • 5.1. Core issue 4: achievement of expected outcomes
  • 5.2. Core issue 5: demonstration of efficiency and economy
  • 5.3. Other evaluation questions
• 6. Conclusions and recommendations
  • 6.1. Conclusions
  • 6.2. Recommendations and management response
• Appendix A: TBS core evaluation issues
• Appendix B: CCI logic model
• Appendix C: evaluation framework
• Appendix D: additional detailed tables and figures
• Appendix E: expert services offered by CCI
• Appendix F: comparative table
• Bibliography

List of tables

• Table 2.1: program profile
• Table 2.2: program outcomes
• Table 2.3: CCI governance structure during the evaluation period
• Table 2.4: key stakeholders and target populations
• Table 2.5: budgeted and actual expenditures – administrative costs ($M)
• Table 3.1: overview of evaluation issues and questions
• Table 3.2: lines of evidence
• Table 3.3: methodological limitations
• Table 5.1: total number of CCI research and development projects initiated and completed, 2010-11 to 2015-16
• Table 5.2: number of events and participants at CCI's advanced professional development workshops from 2010-11 to 2015-16
• Table 5.3: annual number of unique paid and unpaid interns
• Table 5.4: number of print publications distributed and sold, in and outside Canada, 2010-11 to 2015-16
• Table 5.9: type and percentage of institutions served by CCI expert services from 2010-11 to 2015-16
• Table 5.10: regional distribution of services completed by CCI, 2010-11 to 2015-16
• Table 5.11: number of collections serviced by CCI expert services (completed services), 2010-11 to 2015-16
• Table 5.13: staff reductions at CCI, 2010-11 to 2015-16
• Table 5.14: number of days required to complete projects, 2011-12 to 2014-15
• Table 5.15: average difference in days between forecasted and actual number of days required to complete projects, by types of service, 2011-12 to 2014-15
• Table 5.5: top three learning material viewed online (by years), 2010-11 to 2015-16
• Table 5.6: top three CCI notes accessed online, 2010-11 to 2015-16
• Table 5.7: number of completed expert services, by type of services, 2010-11 to 2015-16
• Table 5.8: list of Canadian heritage institutions having received ten or more than ten completed projects, 2010-2011 to 2015-16
• Table 5.12: administrative flexibility of Canada government SOAs (excluding CCI), 2017.

List of figures

• Figure 2.1: CCI’s core activities
• Figure 5.1: number of articles published in peer-reviewed and professional journals, 2010-11 to 2015-16
• Figure 5.2: ratio of number of publications by FTE, 2011-12 to 2015-16
• Figure 5.3: presentations by CCI researchers in Canada and internationally, 2010-11 to 2015-16
• Figure 5.4: average number of participants per regional training (event/workshop), 2010-11 to 2015-166
• Figure 5.5: % of participants to CCI regional workshops by regions, 2010-11 to 2015-16
• Figure 5.6: CCI Web global and online learning traffic, 2010-11 to 2015-16
• Figure 5.7: number of expert services projects completed, 2010-11 to 2015-16
• Figure 5.8: number of unique heritage institutions served, 2010-11 to 2015-16
• Figure 5.9: total number of objects analyzed by CCI, 2010-11 to 2015-16

List of acronyms and abbreviations

ADM

Assistant Deputy Minister

CCI

Canadian Conservation Institute

CCSF

Canada Cultural Spaces Fund

CMA

Canadian Museums Association

COO

Chief Operating Officer

DG

Director General

DPR

Departmental Performance Report

ESD

Evaluation Services Directorate

FTE

Full-Time Equivalent

GBA

Gender-Based Analysis

GIRs

General Information Requests

ICCROM

International Centre for Study of the Preservation and Restoration of Cultural Property

ICOM

International Council of Museums

INDEM

Canada Travelling Exhibitions Indemnification Program

LAC

Library and Archives Canada

MAP

Museums Assistance Program

MCPP

Movable Cultural Property Program

O&M

Operations and maintenance

PAA

Program Alignment Architecture

PCH

Department of Canadian Heritage

PIMS

Preservation Information and Management System

PMERS

Performance Measurement, Evaluation and Risk Strategy

PNR

Prairies and Northern Region

POR

Public opinion research

PRG

Policy Research Group

PSPC

Public Services and Procurement Canada

RCMP

Royal Canadian Mounted Police

RIPEC

Results, Integrated Planning, and Evaluation Committee

RPP

Report on Plans and Priorities

R&D

Research and Development

SOA

Special Operation Agency

SPSS

Statistical Package for Social Sciences

TBS

Treasury Board Secretariat

UNESCO

United Nations Educational, Scientific and Cultural Organization

Executive summary

Overview of the program

CCI advances and promotes the conservation of Canada’s heritage collections through its expertise in conservation science, treatment and preventive conservation. CCI works with heritage institutions and professionals to ensure heritage collections are preserved and accessible to Canadians now and in the future.

CCI became a Special Operating Agency (SOA) on November 19, 1992. This status gives CCI the authority and responsibility to achieve results for Canadians in heritage conservation with a business-oriented corporate culture focused on service delivery. CCI’s mission is in accordance with its program mandate as a SOA which is to advance the practice, science and technology of conservation.

CCI delivers its mandate through 3 core business lines:

• Research and development: CCI performs three types of research, independently or in collaboration with partners: foundation research, applied research and collections preservation research. For collaborative research and development projects, CCI enters into research partnerships with institutions and organizations both in Canada and internationally in order to advance its research and conservation practices.
• Knowledge dissemination: CCI disseminates conservation knowledge to Canadian heritage institutions and professionals so that they have the information, data, tools, and skills to care for the collections which have been entrusted to them. In addition, CCI’s knowledge is shared through training, online resources and print publications to heritage professionals and workers in other countries.
• Expert services: CCI expert services include treatments, preventive conservation, and scientific analysis to Canadian and foreign heritage institutions and clients.

CCI’s expected results are the following:

Immediate outcomes:

• Heritage institutions and workers take advantage of CCI learning opportunities;
• New knowledge in heritage preservation and conservation is created; and
• Heritage institutions use expert services to preserve and better understand their collections.

Intermediate outcomes:

• Heritage institutions and workers improve their professional knowledge, skills and practices to preserve their collection;
• New knowledge in heritage preservation and conservation is disseminated nationally and internationally; and
• Heritage cultural objects are preserved and better understood.

Ultimate outcomes:

• Heritage institutions and workers are preserving their collections; and
• CCI Services enable heritage institutions to reach their goals of preservation and accessibility.

PCH strategic outcome:

Within PCH, CCI is one of five program sub-activities under the Heritage program activity that helps the department meet its strategic outcome according to which: “Canadian artistic expressions and cultural content are created and accessible at home and abroad.”

Total CCI expenditures for the period covered by the evaluation were $60.4M. Reductions in budgets and expenditures occurred throughout the evaluation period, from $12.4M in 2010-11 to $7.0M in 2015-16, accounting for a 40% overall reduction in budget.

Evaluation approach and methodology

The evaluation covered the period from 2010-11 to 2015-16, and was designed and conducted in accordance with the Treasury Board Secretariat (TBS) Policy on Evaluation (2009) and other components of the TBS evaluation policy suite. In accordance with the Directive on the Evaluation Function (2009), the evaluation addresses the five core evaluation issues relating to the relevance and performance (effectiveness and efficiency) of the CCI.

The evaluation approach involved a combination of qualitative and quantitative data collection methods designed to address the evaluation issues and questions as presented in the Evaluation Matrix. More specifically, lines of evidence included: 1) literature review, 2) bibliometric assessment, 3) review of documentation, administrative data, databases and files, 4) case studies, and 5) interviews with key informants. Both primary and secondary data sources were therefore used for the evaluation. The use of multiple lines of evidence provided the opportunity to develop findings based on a triangulation of evidence, contributing to the strength of conclusions.

Results

Relevance of CCI

There is a strong need for CCI to continue to support heritage institutions and professional workers to preserve and protect Canada’s priceless heritage. The limited funding and competencies of most heritage institutions and the need to improve the conservation of collections as well as the restoration of objects underlines the continued need of CCI. Federal intervention is therefore needed because many heritage institutions would not otherwise have access to private preservation and conservation expertise and services.

CCI is aligned with federal government priorities as outlined in the Speech from the Throne of 2015 outlining that the Canadian Conservation Institute contributes to the Canada 150 priority by prioritizing conservation treatments of artifacts that support the 2017 celebrations. More recently, budget 2016 affirmed Canada’s support to preserve and protect priceless Canadian heritage artifacts.

There is a role for the federal government in supporting heritage institutions and professional workers in the preservation and conservation of cultural objects and collections. According to external key informants, CCI’s role and mandate are clear and well understood. The evaluation compared the mandates and missions of key players involved in heritage conservation and no duplication or overlapping was identified when considering CCI mandate and activities. CCI distinguishes itself by the role it imparts to conservation Research and development in the fulfilment of its mandate and the support it brings to heritage institutions in Canada by offering a comprehensive suite of expert services that is not limited to conservation treatment.

CCI’s operational context

During the evaluation period, a number of government decisions had a direct impact on the Institute’s capacity to meet its objectives and achieve its expected results. 

The deficit reduction measures, announced in the 2010 Budget, aimed at reducing the rate of growth of the federal government departments and agencies operating expenditures. One of the measures of the Budget was the important limitation to available funding for travel.  This limitation had impacts both in terms of CCI's researchers’ capacity to attend national and international fora related to heritage conservation and in terms of opportunities to directly interact, discuss and partner on collaborative Research and development projects. These measures which were continued the following years under the Deficit Reduction Action Plan also structurally impacted CCI:

• to increase efficiency in the delivery of programs, PCH centralized most of its internal services (human resources, finance and information service management) leaving CCI less able to directly address its unique needs, for example, in specialized human resources recruitment or scientific computing;
• CCI's direct budget was reduced by approximately 40%;
• CCI underwent a reduction of one quarter (25%) of its positions (of which one third were related to the transfer of internal services positions) to reduce overall costs.

Performance

CCI adapted to its operational context and acted on previous evaluation recommendations by developing strategic, research and professional development plans, which had an impact on overall activities of the Institute.

Research and development: CCI’s capacity to create new heritage preservation and conservation knowledge has diminished in recent years.

• The number of research and development projects completed decreased during the evaluation period, and CCI reduced importantly the number of new projects initiated during the last 3 years:
  • more than two thirds (67%) of the research and development projects were completed between 2010-11 and 2012-2013;
  • out of eleven research and development projects still active at CCI, nine were approved before or during 2012-13 and only two projects have been approved in 2013-14 or later.
• CCI researchers have published fewer articles in scientific peer-reviewed and professional journals, a reduction by more than half, going from a high 19 publications in 2011-12 to a low 6 publications in 2015-16.
• Presentations to peers have kept relatively the same level of intensity.
• The dwindling national and international presence of CCI has been remarked upon by its usual national and international collaborators: 2014-15 travels (9 trips) were reduced by 70% in comparison with 2010-2011 number (29 trips). 

Knowledge dissemination: CCI has maintained the intensity level of most of its much appreciated training activities.

• Data collected show that a diversity of professionals and institutions from all parts of Canada are taking advantage of CCI's learning opportunities and have therefore improved their professional knowledge through these opportunities.
• CCI has increased its knowledge dissemination activities through an increase in resources available on its web and Facebook pages and by updating and increasing the number of specialized conservation notes available on its website. The conversion of CCI website to the generic Government of Canada website has had an impact on accessibility.

Expert services: CCI has reduced by 29% the number of expert services provided to the Canadian heritage community, resulting in a diminishing number of heritage institutions being served annually.

• There is evidence that a variety of different heritage institutions across Canada are using a broad range of expert services offered by CCI. Between 2010-11 and 2015, CCI completed 998 expert services projects, a yearly average of 166 services completed.
• The total number of unique heritage institutions served has been decreasing over the last 6 years, from 111 institutions for which request were accepted and completed in 2010-11, to 75 by 2015-16.
• CCI has restored or supported the analysis of a large number of heritage objects, and has supported the preventive conservation of numerous collections of such objects.
• Users of CCI’s expert services have reported high levels of satisfaction for the services provided, mentioning that CCI expert services are of quality, are aligned with their needs and are useful, considering that oftentimes they do not have the expertise to do the work themselves.
• Users have also indicated that all three types of expert services provided by CCI (conservation treatment, preventive conservation and scientific analysis) help heritage institutions better understand their objects and collections.
• Users have also expressed some concerns about the length of time it takes for CCI to treat heritage objects or artifacts and about its capacity to provide timely updates/follow-ups/responses to enquiries.

Efficiency

CCI has been affected considerably by the changes brought to its operational context and its ability to continue to provide timely expert services, including its capacity to plan the delivery of its services. Even though CCI has a Special Operating Agency status that should provide it with flexibility, the centralization of corporate services has removed a large part of that flexibility, hampering its service-oriented nature.

• An examination of the number of days it took CCI to complete projects in the three expert services aimed specifically at the Canadian heritage community and over which CCI has control shows that it experienced difficulties in delivering its services, with an average of 80 days required to complete all services in 2011-12 that increased to an average of 194 days in 2013-14, before falling to an average of 133 days in 2014-15.
• The evaluation also looked at the data available concerning the forecasted and actual time needed to complete expert services projects. In 2011-12, the projects were completed in an average of 9.3 days later than forecasted, which climbed to an average of 73.7 days per project in 2013-14, and 45.4 days in 2014-15.

Performance information is captured by CCI. Nevertheless, some limitations in the performance measurement strategy have been identified:

• The methodology of some performance measures need to be reviewed to ensure that they accurately reflect the activities and results of CCI, especially indicators important to measure evolution over time;
• The methodology used for CCI’s client surveys must be reviewed, documented and implemented (sampling of clients who receive expert services, measurement of behavior changes (practices) as a result of CCI’s training);
• Data on service standards as posted on CCI’s website is not captured;
• Performance information is not and cannot be well integrated into CCI’s decision-making process with the current reporting capabilities of CCI’s operating system (PIMS).

Recommendations and management response

Action plan

Action item	Deliverable(s)	Timeline	Program official responsible
1.1 Identify specific administrative governance structures and processes which negatively affect CCI’s capacity to fulfill its mandate.	Analysis of opportunities to maximize administrative flexibility, with options for consideration.	September 30, 2017	CCI DG
1.2 Enter into discussions with the relevant authorities in order to identify preferred options.	Preferred options to be presented to the DM for approval.	December 31, 2017	CCI DG
1.3 Develop plan and implement measures approved by the Deputy Minister.	Implementation plans and implementation of identified transfers in administrative processes and related resources.	December 31, 2018	CCI DG

Full implementation date
December 31, 2018

Action plan

Action item	Deliverable(s)	Timeline	Program official responsible
2.1 Gather data collection, reporting and usability requirements to enhance the capacity of CCI’s information management system to support business intelligence and reporting activities.	Business Requirements Document	October 30, 2017	CCI DG
2.2 Develop a plan for the Preservation Information Management System’s (PIMS) next phase of development.	Project Agreement with the Chief Information Officer Branch	March 31, 2018	CCI DG
2.3 Implement measures specifically addressing identified data integrity gaps and reporting capacity.	Stage one of PIMS enhancements, as identified in the Project Agreement, is completed.	September 30, 2018	CCI DG
2.4 Fully implement the Preservation Information Management System’s (PIMS) next phase of development.	Remaining PIMS enhancements, as identified in the Project Agreement, are completed.	March 31, 2022, contingent upon CIOB’s capacity to complete the deliverables by this date.	CCI DG

Full implementation date
March 31, 2022

1. Introduction

1.1. Purpose

The report presents the findings and recommendations from the 2010-11 to 2015-16 evaluation of the Canadian Conservation Institute (CCI). The evaluation of the CCI was conducted to provide comprehensive and reliable evidence to support decisions regarding the relevance and performance of the Institute. The evaluation was conducted between April 2016 and January 2017.

The evaluation report provides information and context for the CCI, the evaluation methodology and the findings for each evaluation question, as well as overall conclusions and recommendations. The evaluation was conducted as required by the 2016-17 to 2020-21 Departmental Evaluation Plan. The evaluation was led by the Evaluation Services Directorate (ESD) of Department of Canadian Heritage (PCH) with contributions from the PCH Policy Research Group (PRG) for the literature review component, as well as consulting firms for a) the bibliometric assessment of CCI peer-reviewed publications, and b) the review of the Special Operation Agency (SOA) model.

The evaluation covered the period from 2010-11 to 2015-16, and was designed and conducted in accordance with the Treasury Board Secretariat (TBS) Policy on Evaluation (2009) and other components of the TBS evaluation policy suite. In accordance with the Directive on the Evaluation Function (2009), the evaluation addresses the five core evaluation issues relating to the relevance and performance (effectiveness and efficiency) of the CCI. The five core evaluation issues can be found in Appendix A.

The report is structured as follows:

• Section 2 presents an overview of the CCI;
• Section 3 presents the methodology employed for the evaluation and the associated limitations;
• Section 4 presents the findings related to the evaluation issue of relevance;
• Section 5 presents the findings for performance (including those related to achievement of outcomes and efficiency/economy); and
• Section 6 presents the conclusions and recommendations.

2. Program profile

2.1. Background

Table 2.1 outlines the key program profile elements.

Table 2.1: program profile

Program inception	CCI was created in 1972 after Canada became a signatory to the UNESCO Convention on the Means of Prohibiting and Preventing the Illicit Import, Export and Transfer of Ownership of Cultural Property (1970).Footnote 1 Canada set up a system to protect cultural property, and established structures to support the preservation of cultural property such as national inventories of protected property, conservation institutions and a cultural property export control system.
Mission	CCI advances and promotes the conservation of Canada’s heritage collections through its expertise in conservation science, treatment and preventive conservation. CCI works with heritage institutions and professionals to ensure heritage collections are preserved and accessible to Canadians now and in the future.
Delivery model	CCI became a SOA on November 19, 1992. This status gives CCI the authority and responsibility to achieve results for Canadians in heritage conservation with a business-oriented corporate culture focused on service delivery. CCI’s mission is in accordance with its program mandate as a SOAFootnote 2 which is to advance the practice, science and technology of conservation.
Delivery mechanisms	Research and development: CCI performs three types of research, independently or in collaboration with partners: foundation research, applied research and collections preservation research. For collaborative research and development projects, CCI enters into research partnerships with institutions and organizations both in Canada and internationally in order to advance its research and conservation practices. Knowledge dissemination: CCI disseminates conservation knowledge to Canadian heritage institutions and professionals so that they have the information, data, tools, and skills to care for the collections which have been entrusted to them. In addition, CCI’s knowledge is shared through training, online resources and print publications to heritage professionals and workers in other countries. Expert services: CCI expert services include treatments, preventive conservation, and scientific analysis to Canadian heritage institutions and clients.

Program theory and program outcomes

CCI's activities in all three of its delivery mechanisms (research and development, expert services and knowledge dissemination) are linked and integrated. However, research and development activities are at the core of CCI's purpose and success as the results of research are integrated into CCI services, are also disseminated to CCI's peers through publication in professional journals, and are incorporated into CCI's publications, online information, and training workshops. This approach requires that conditions be present to ensure the full implementation of the program theory:

• Ability for the Institute to innovate, collaborate and co-operate in an interdisciplinary way with Canadian and international institutions dedicated to the conservation of heritage;
• That the availability of human resources and financial context enable the full participation of CCI in research projects that support responses to the heritage community needs, therefore enhancing the implementation of the program theory.

CCI's outcomes as illustrated in its logic model (Appendix B), are based on the full implementation of the program theory.

Figure 2.1: CCI’s core activities – text version

Heritage institutions and professionals
Research and development
Expert services
Knowledge sharing

Table 2.2 outlines the program’s intended outcomes. The logic model used for the evaluation was enhanced conjointly by CCI and the ESD in 2016 to reflect all the activities of CCI.

Table 2.2: program outcomes

PCH strategic outcomes	Within PCH, CCI is one of five program sub-activities under the Heritage program activity that helps the department meet its strategic outcome according to which: “Canadian artistic expressions and cultural content are created and accessible at home and abroad.” CCI conducts research on significant heritage objects and collections, as well as emerging needs and threats to collections. research and development and expert services provided feed and support the dissemination of knowledge, technique and tools to support the long-term accessibility of Canada’s heritage.
Ultimate outcomes	Heritage institutions and workers are preserving their collections; and CCI Services enable heritage institutions to reach their goals of preservation and accessibility.
Intermediate outcomes	Heritage institutions and workers improve their professional knowledge, skills and practices to preserve their collection; New knowledge in heritage preservation and conservation is disseminated nationally and internationally; and Heritage cultural objects are preserved and better understood.
Immediate outcomes	Heritage institutions and workers take advantage of CCI learning opportunities; New knowledge in heritage preservation and conservation is created; and Heritage institutions use expert services to preserve and better understand their collections.

2.2. Program management, governance, target groups, key stakeholders and partners

The Institute’s governance structure ensures that decision-making is focussed on achieving results and fulfilling corporate objectives, and that its control management (assets, money, people and services) is integrated and effective, and is in line with the PCH governance structure.

Table 2.3 outlines CCI’s governance structure during the evaluation period.

Table 2.3: CCI governance structure during the evaluation period

CCI executive committee	CCI’s decision-making body for strategic issues. It is chaired by the Chief Operating Officer (COO) and Director General (DG) and its members include the Associate DG / Director of Research, Conservation and Scientific Services (Alternate chair), the Director of Corporate Services and Real Property, and the Director of Client Relations and Professional Development. The Executive Committee provides oversight so that CCI achieves its strategic priorities and operational and business objectives.
Associate DG / director of research, conservation and scientific services	Is accountable for providing scientific and technical leadership and strategic directions for the Institute’s international calibre research and development program; disseminating research results in Canada and internationally; managing the delivery of a comprehensive package of expert services to clients in Canada; and directing CCI’s training and advanced professional development programs.
The director client relations and professional development	Is responsible for CCI’s publishing services, including printed and web publications, and for professional development activities, including training materials, workshops, advanced professional programs.
Director of corporate services and real property	Is the Director of finances responsible for the management of all internal services (finance, HR, etc,) and administrative support, including the management of the real property and its installations, before it was transferred to central services.
Management committee	Is comprised of the DG, Directors and Managers and meets regularly to discuss and make decisions on business and client-related issues and policies related to the research and development program, services to clients, and knowledge dissemination activities.

A rebalancing of CCI’s organizational structure occurred in January 2016 with the administrative merger of CHIN and CCI. Membership in the CCI Executive Committee was adjusted as follows: Director General (Chair), Associate DG / Director of Research, Conservation and Scientific Services (Alternate chair), Director of Heritage Information, and Manager of Business Innovation. The Director of Heritage Information is to a great extent responsible for the same activities that fell under the Director of Client Relations and Professional Development in the table 2.3., in addition to the newly integrated CHIN business activities. The Manager of Business Innovation is responsible for many of the functions that were previously overseen by the Director of Corporate Services and Real Property, but the scope of responsibilities changed significantly given the various centralization exercises (e.g. real property).

The main beneficiaries of CCI delivery mechanisms are heritage institutions and professional workers in preservation and conservation science. Table 2.4 presents the key stakeholders and target populations.

Table 2.4: key stakeholders and target populations

Target population The specific individuals or organizations intended to be influenced and benefit from the program.	Canadian museums or organizations whose role is to acquire, conserve, research, communicate, and exhibit (for purposes of study, education, and enjoyment) permanent heritage collections. Museums Archives Libraries Historic sites
Key stakeholders Agencies, organizations, groups or individuals who have a direct or indirect interest in the program intervention or its evaluation.	Municipal, provincial / territorial agencies or government (e.g. Centre de conservation du Québec) Federal government institutions responsible for heritage collections, including the Public Services and Procurement Canada (PSPC) - Heritage Conservation Directorate, the Department of National Defence and Agriculture Canada Department of Canadian Heritage Programs: INDEM, MCPP, MAP and CCSF RCMP and Interpol for artefact authentication Educational Institutions Aboriginals communities National and international organizations (CMA, ICCROM, ICOM, professional associations, etc.)
Partners	CCI establishes partnerships and collaborative relationships to undertake joint projects related to research and knowledge dissemination with Canadian and international institutions such as: Library and Archives Canada Parks Canada Provincial Museum Associations, Canadian Museum Association, Canadian Association for Conservation, Canadian Council of Archives, and other provincial and regional heritage organizations The Getty Conservation Institute, Los Angeles, U.S. ICCROM, Rome National Research Institute for Cultural Properties of Japan

2.3. Program resources

Total CCI expenditures for the period covered by the evaluation were $60.4M. Table 2.5 presents the budgeted and actual administrative expenditures for CCI during the period covered by the evaluation. Reductions in budget and expenditures occurred throughout the evaluation period, from $12.4M in 2010-11 to $7.0M in 2015-16, accounting for a 40% reduction in overall budget.

Table 2.5: budgeted and actual expenditures, 2010-11 to 2015-16

Dollars	2010- 2011	2011- 2012	2012- 2013	2013-2014	2014- 2015	2015-2016	Total	Ongoing
Vote 1 (operating expenditures)	6,529,276	6,216,797	5,952,625	5,228,570	5,193,748	5,137,546	34,258,562	5,137,546
EBP	1,305,855	1,243,359	1,190,525	1,045,714	1,038,750	1,027,509	6,851,712	1,027,509
O&M	5,045,915	4,319,990	4,216,613	3,745,879	3,648,198	1,541,371	22,517,966	1,541,371
VNR Footnote 3	-500,000	-500,000	-500,000	-500,000	-500,000	-700,000	-3,200,000	-700,000
Total program Footnote 4	12,381,046	11,280,146	10,859,763	9,520,163	9,380,696	7,006,426	60,428,240	7,006,426

Source: Resource Management Directorate - Canadian Conservation Institute

3. Evaluation methodology

3.1. Scope of the evaluation, calibration and quality control

Scope of evaluation

The evaluation’s objective is to provide credible and neutral information on the core evaluation issues of relevance and performance, including effectiveness and efficiency of CCI, in accordance with the 2009 Treasury Board Directive on the Evaluation Function (see Appendix A). The evaluation covered a period of six (6) years starting from April 1, 2010 to March 31, 2016. The evaluation was led by the ESD at PCH. The PRG led the development of the literature review component.

Calibration

The overall approach to the evaluation has taken into consideration the results of a calibration exercise conducted by ESD, in an effort to conduct a quality evaluation study in a cost-effective manner. Calibration refers to the process of adjusting how evaluations are conducted based on a number of different factors such as the scope, the approach and design, the data collection methods, reporting and/or project governance and management - while maintaining the credibility and usability of the evaluation results.

The evaluation of CCI was calibrated as follows:

• Whenever possible, ESD used existing information (e.g. information on the results / performance measurement data collected) to assess the evaluation issues and gather new data when necessary.
• The number of key information interviews was reduced and sampling was used to determine the number and type of key informants.
• The evaluation focused on core as well as key issues.

Quality control

The quality of the evaluation was ensured through senior-level ESD staff conducting the planning of the evaluation, including the approval of the Terms of Reference for the evaluation by PCH’s Results, Integrated Planning, and Evaluation Committee (RIPEC). During the conduct of the evaluation, ESD developed, collected and analyzed the data originating from all lines of evidence, as well as conceptualized, reviewed and approved the evaluation data collection tools and approaches for the literature review conducted by PRG, as well as the bibliographic assessment and the review of the SOA model that were conducted by external firms. The draft evaluation report was reviewed by the program’s senior management.

3.2. Evaluation questions by issue area

The evaluation addresses the five core issues of relevance and performance as outlined in the TBS Directive on the Evaluation Function (2009). The evaluation also examined the program’s design and delivery, areas for improvement and performance measurement.

The evaluation questions for the evaluation, and associated indicators, were selected based on the CCI logic model. The questions and associated indicators by core issue, as well as the data sources and collection methods are set out in the evaluation matrix, found in Appendix C. Table 3.1 presents the evaluation issues and questions addressed in the evaluation of the CCI.

Table 3.1: overview of evaluation issues and questions

Relevance

Issues	Questions
1) Continued need for the program	1a) How does CCI serve the needs of Canadians? 1b) Is CCI relevant for heritage institutions and workers in preserving Canada’s heritage collections? 1c) Is CCI responsive to the needs of Canada’s heritage institutions?
2) Alignment with government priorities	2) To what extent are CCI objectives aligned with governmental priorities and the Department’s strategic objectives?
3) Consistency with federal roles and responsibilities	3) To what extent is CCI aligned with the federal government’s roles and responsibilities?

Performance - effectiveness

Issues

Questions

4) Achievement of expected outcomes

4a) To what extent did CCI reach the immediate outcome that heritage institutions and workers take advantage of CCI learning opportunities? 4b) To what extent did CCI reach the immediate outcome of creation of new heritage preservation and conservation knowledge? 4c) To what extent did CCI reach the immediate outcome of heritage institutions using expert services to preserve and better understand their collections 4d) To what extent did CCI reach its intermediate outcome that heritage institutions and workers improve their professional knowledge, skills and practices to preserve their collection? 4e) To what extent did CCI reach its intermediate outcome that new heritage knowledge is made available (and used Footnote 5) nationally and internationally? 4f) To what extent did CCI reach its intermediate outcome of heritage cultural objects and collections are preserved and better understood? 4g) To what extent did CCI reach its ultimate outcome that heritage institutions and workers are preserving their collections? 4h) Are CCI services enabling heritage institutions to reach their goals of preservation and accessibility?

Performance and efficiency

Issues

Questions

5) Demonstration of efficiency and economy

5a) Is the current organizational model (SOA) still appropriate for CCI? 5b) To what extent is CCI delivered efficiently? 5c) Were adequate management and administrative practices in place for effective service delivery? 5d) To what extent does CCI duplicate, overlap or complement other programs in Canada? 5e) What is the impact of the revenue generating services on CCI services that are offered for free?

Other

Issues	Questions
6) Other evaluation issues(s)	6a) Is the current performance measurement framework effective at capturing the results of CCI? 6b) Were all official language requirements met by CCI? 6c) Has gender-based analysis (GBA+)Footnote 6 been considered? 6d) Are there any unexpected results identified by the evaluation?

3.3. Evaluation methods

The evaluation approach involved a combination of qualitative and quantitative data collection methods designed to address the evaluation issues and questions as presented in the Evaluation Matrix. Both primary and secondary data sources was used for the evaluation. The use of multiple lines of evidence provided the opportunity to develop findings based on a triangulation of evidence, contributing to the strength of conclusions. The findings from each line of evidence were triangulated and are presented in the final report.

Lines of evidence

Data collection and reporting of preliminary evaluation results were undertaken between April 2016 and January 2017. Table 3.2 outlines the four lines of evidence used for the evaluation of CCI

Table 3.2: lines of evidence

Literature review	The literature review provided a summary and an analysis of research from recently published literature, reports, articles, websites, academic journal databases, publications, public opinion research (POR) and other sources on the subject matter related to a variety of questions identified in the project’s evaluation matrix.
Bibliometric assessment	A private firm conducted a bibliometric assessment of CCI peer-reviewed publications.
Review of documentation, administrative data, databases and files	Federal, departmental and program documents were analyzed to assess the relevance and performance of CCI. A review and analysis of the program’s administrative data, files and database was also conducted to provide quantitative and qualitative information on CCI activities and their results.
Case studies	A total of 18 case studies designed to assess the impact of CCI’s expert services on heritage communities and the public were conducted. The case studies involved document and file reviews, including interviews to gather information on results and impacts. A case study summary was developed to support analysis and triangulation.
Interviews with key informants	In-depth interviews were conducted with key informants (N=27). The purpose of the key informant interviews was to gain a better understanding of the perceptions of individuals who have a significant knowledge, role or experience in the design and/or delivery of CCI. Key informants included CCI personnel, PCH partners, domestic and international experts in preservation and conservation, recipients of CCI expert services, participants in CCI training, collaborators in CCI research and development projects, as well as external partners. The key informant interviews took place either in-person or by telephone.

Methodological limitations

The limits regarding methodology for the evaluation of CCI are outlined in table 3.3.

Table 3.3: methodological limitations

CCI’s client satisfaction surveys	The methodology used by CCI to conduct surveys limits their usefulness for the purposes of the evaluation. Limitations include: inconsistent methodology, including unknown consideration used to solicit clients to participate in the survey, and difficulties in measuring change of behaviour after training (skills, practice). Mitigation strategies included an increased number of interviews and specific questions in the case studies.
Multiple sources of program data	CCI standardized reports in Preservation Information and Management System (PIMS) use is limited as it does not offer the flexibility to select all the data elements necessary for analysis. Mitigation included downloading PIMS data in Excel files and analysis with SPSS, as well as using the CCI Indicator Reports. While proportions are similar across the 3 data sources, numbers can vary. The choice of the data source is based on ESD’s evaluation of the quality and validity of the information available.
CCI’s performance measurement strategy	A number of indicators - “transactions”, “number of research and development projects”, “number of partners for research and development” - do not provide complete and adequate information about certain activities undertaken by the Institute.

4. Findings - relevance

The following sections present the key evaluation findings related to relevance.

4.1. Core issue 1: continued need for the program

Continued need for the Canadian Conservation Institute

Canadians visit museums and other heritage institutions housing thousands of permanent and temporary exhibits, as well as millions of artifacts/cultural objects and collections

Art galleries, museums, historic sites, and archives saw an increase in the number of visits they received during 2013, with overall attendance reaching a record 61.9 million visits, up 10.4% from the 2011 figures (56 million). Museums received the majority of those visits, topping 25 million, followed by historic sites with 11.7 million. Heritage institutions house over 60 million artifacts and objects, which include archeological artifacts, military objects, fine art, scientific or technological objects, and more. Museums are responsible for protecting over 82% of these treasures, followed by historic sites at 16%. Exhibiting artifacts, works of art and other types of collections is a form of storytelling. Heritage institutions create, host and circulate exhibitions to teach and inspire. In 2013 there were over 16,000 permanent exhibitions in Canada – a figure that is similar to the 2011 figures. Newly created exhibitions (which include new permanent exhibitions, temporary non-travelling exhibitions and travelling exhibitions) climbed slightly from 7,703 produced in 2011 to 7,838 produced in 2013, a rise of approximately 1.7%. ^{Footnote 7}

The benefits of preservation and conservation of heritage objects and artifacts to ensure public access are well-described in the literature

Bringing together persons from all walks of life to work on a common project or objective, contributes to “social cohesion” or “social capital” – for example, by increasing social trust, reciprocity and a sense of belonging in communities Social benefits of cultural events can include creating a sense of place. ^{Footnote 8}

POR shows consensus among Canadians concerning the benefits of arts and culture and heritage to communities

Large majorities either “strongly” or “somewhat” agree that arts and culture and heritage is a valuable way of bringing together people from different languages and cultural tradition; that it “makes a community a better place to live”, that it is important to individual wellbeing”, that it “helps us express and define what it means to be Canadian”, and that it is important to a community's economic wellbeing”.^{Footnote 9}

The conditions which led to federal support still exist

The key factor underpinning the creation of CCI in 1972 was that Canada’s cultural heritage was in a state of neglect. ^{Footnote 10}A central conservation institute is still needed in order to advise Canadian heritage institutions on proper conservation practices to protect Canada’s cultural heritage. Key informants, including international experts, also expressed the need for federal intervention in preservation and conservation of cultural heritage, with some indicating that the CCI model should be adopted in other countries.

The limited funding available for heritage institutions and the need to improve the conservation of collections as well as the restoration of objects underlines the continued need of CCI

Consultations undertaken by CCI following the 2012 Evaluation identified five major challenges facing heritage institutions: 1) Managing facilities: storage, environmental conditions; 2) Ensuring safe access to collections; 3) Managing electronic and digital collections; 4) Understanding/conserving increasingly diverse collections, both contemporary and traditional materials; and 5) Accessing/securing conservation and technical expertise. CCI’s current activities and future direction, as described in their strategic plan of 2015-2020, are aligned with the needs and challenges facing heritage institutions regarding preservation and conservation.

Results of the case studies and key informant interviews indicate that in the absence of CCI, heritage institutions would not have the expertise to conduct the work, and would also not have the resources (financial and/or human) to conduct the work themselves. Federal intervention is needed because many heritage institutions would not otherwise have access to private preservation and conservation expertise and services.

Emerging issues

Results from key informant interviews indicate that new issues are also emerging. More specifically, regarding the arrival of new technologies and practices in conservation, as well as the impact of climate change on conservation practices. CCI’s strategic plan 2015-2020 is aligned with the need to adapt to new technologies and practices in conservation. CCI will need to develop approaches to address the impact of climate change and therefore better meet the needs of stakeholders.

Scholars have also expressed concern about the environmental impacts of conservation.^{Footnote 11} In particular, many experts are noting that climate change might have an impact on how built heritage and artefacts are stored and preserved.^{Footnote 12} For example, rising sea levels might have an impact on built heritage or artefacts as most cities are built near or along waterways. Flooding and similar natural disasters like hurricanes and forest fires have been tied to climate change, and some conservation professionals have considered the impact these disasters might have on heritage infrastructure and objects. The 2015 Canadian Association for Conservation of Cultural Property Annual Conference had an entire section of panels on disaster management covering conservation issues from floods to asbestos.^{Footnote 13}

4.2. Core issue 2: alignment with government priorities

Alignment with Government of Canada priorities

CCI is aligned with federal government priorities as outlined in the Speech from the Throne of 2015 ^{Footnote 14} outlining that the Canadian Conservation Institute contributes to the Canada 150 priority by giving priority to conservation treatments of heritage objects and artefacts that support the 2017 celebrations. CCI has in fact already started, in 2012, working on different significant objects related to the road to the 150th anniversary of the Canadian federation and the anniversary itself.

The Budget of 2016 ^{Footnote 15} also demonstrated support of the federal government for the preservation and protection of priceless Canadian heritage artifacts through the possible investment of up to $156 million for the construction of a new Collection and Conservation Centre. This major infrastructure project will be realized by PCH in collaboration with the Canada Science and Technology Museums Corporation, as well as the National Art Gallery.

CCI also contributes to Canada's Northern Strategy by offering services (training, facilities consultations and northern archeology) to Northern communities’ cultural organizations.

Alignment with PCH priorities and strategic outcomes

CCI is aligned with PCH’s strategic outcome: "Canadian artistic expressions and cultural content are created and accessible at home and abroad", and aligns with PCH priorities with Canada 150, and contributed to the priorities around the War of 1812. CCI has also demonstrated flexibility by adjusting its selection criteria in response to the Government of Canada budget priorities relating to history (e.g. Canada 150).

4.3. Core issue 3: alignment with federal roles and responsibilities

While CCI itself has no direct basis in legislation, several related legislative instruments affirm the Government of Canada’s role, and in particular PCH’s role, in the care and preservation of heritage collections. The Canadian Heritage Act ^{Footnote 16} recognizes the Heritage Minister’s jurisdiction over “conservation, exportation and importation of cultural property”. The Museums Act ^{Footnote 17} declares that “the heritage of Canada and all its peoples is an important part of the world heritage and must be preserved for present and future generations.” Canadians recognize the delivery of programs and activities that promote arts and heritage across Canada as a legitimate role for the federal government. The Library and Archives of Canada Act ^{Footnote 18} states that “it is necessary that the documentary heritage of Canada be preserved for the benefit of present and future generations”.

Within the federal government, CCI assists departments and agencies in implementing the Treasury Board Policy on Management of Materiel (2006) as it applies to the management of moveable heritage assets. Under this policy, deputy heads are required to ensure that heritage collections are identified and protected. CCI is identified in the policy as the centre of excellence in heritage preservation and the resource for advice, assistance and information on preservation and conservation serving Canadian government departments and agencies.

PORs show that a large majority of Canadians agree that the federal government should help “protect and preserve Canada's heritage” (95%); provide “support for arts and culture in Canada” (90%); and partner “with others to ensure that there are enough arts and cultural facilities to serve the public” (89%). ^{Footnote 19}

Most external key informants interviewed indicated that the federal government has an important role to play in the conservation and preservation of heritage objects, and most agreed the CCI’s role and mandate are clear and well understood.

5. Findings - performance

The following sections present the key evaluation findings related to performance: effectiveness and efficiency.

Context 2010-11 to 2015-16 and structural impacts on CCI

During the evaluation period, a number of government decisions had a direct impact on the Institute’s capacity and required it to adapt to keep meeting its main objectives and achieve its expected results. 

The deficit reduction measures, announced in the 2010 Budget, aimed at reducing the rate of growth of the federal government departments and agencies operating expenditures. One of the measures of the Budget was the important limitation to available funding for travel.  This limitation had impacts both in terms of CCI's researchers’ capacity to attend national and international fora related to heritage conservation and in terms of opportunities to directly interact, discuss and partner on collaborative research and development projects (2014-15 travels number (9 trips) were reduced by 70% in comparison with 2010-2011 number (29 trips).  These measures which were continued the following years under the Deficit Reduction Action Plan also structurally impacted CCI:

• to increase efficiencies in the delivery of programs, PCH centralized most of its internal services (human resources, finance and information service management) leaving CCI less able to directly address its unique needs, for example, in specialized human resources recruitment or scientific computing;
• CCI's direct budget was reduced by approximately 40%, including the transfer of funds for internal/corporate services and accommodation;
• CCI underwent a reduction of more than one quarter of its human resource positions (of which half were related to the transfer of internal services positions) to reduce overall costs.

5.1 Core issue 4: achievement of expected outcomes

Considering the importance of the program design for the achievement of outcomes, the presentation of the outcome findings proceeds using the three pillars of the program: research and development, knowledge dissemination and expert services and presenting for each the findings related to their respective immediate and intermediate outcomes.

The creation of new heritage preservation and conservation knowledge is central in CCI's program design, objectives and outcomes. CCI works at creating knowledge through diverse types of activities: research and development projects which aims at developing advanced techniques for treatment, as well as practical and innovative solutions for caring for collections, scientific expertise which provides detailed information on the materials and structure of artifacts, works of art or heritage collections and through conservation treatment (restoring artifacts and works of art to prevent further deterioration, aid interpretation, or re-establish culturally significant qualities. Treatments can range from minimal stabilization to extensive restoration or reconstruction. This section will address the first type of knowledge creation, research and development.

1. Research and development

Although initiating and contributing to research and development per se is important, the results of these projects are slated to be translated and transferred to the conservation community to ensure its use. This is often (but not uniquely) done through writing articles that are published chiefly in conference proceedings, as well as professional journals.

Achievement of immediate outcome: creation of new heritage preservation and conservation knowledge

Evaluation findings show that CCI has been creating new heritage knowledge during the 6 years covered by the evaluation. CCI researchers:

• completed 39 research and development projects related to the fields such as organic chemistry/biochemistry, chemistry, photonics, physics and collection protection that created knowledge supporting the conservation treatment of paper, wood, plastics and the safe display of objects and collections. A third of the projects included collaborations with Canadian governmental, public and private organizations or international establishments (Japan and USA).
• published 82 articles in professional practices and peer-reviewed journals.

The evaluation findings also shows that there is a clear tendency to reduction of both research and development projects and publications in peer-reviewed journals, thus resulting in fewer opportunities to create new knowledge. research and development and published articles are directly connected together as publications are dependent on the finalization of research and development projects.

Findings have identified that:

• more than two thirds (67%) of the research and development projects were completed between 2010-11 and 2012-2013;
• out of eleven research and development projects still active at CCI, nine were approved before or during 2012-13 and only two projects have been approved in 2013-14 or later.

Table 5.1: total number of research and development projects initiated and completed, 2010-11 to 2015-16

Research and development projects	2010-2011	2011-2012	2012-2013	2013-2014	2014-2015	2015-2016	Total
New approved	21	1	5	13	2	0	42
Completed	10	6	10	7	4	2	39

Source: PIMS standardized reports, April 2017.

A bibliometric assessment examined the relative citation score used to measure paper-level impact, and the relative impact factor, used to measure journal-level impact. The bibliometric analysis shows that CCI’s peer-reviewed articles have received less citations than publications in the same disciplinary fields (mostly analytic chemistry and less frequently, information and library sciences and archaeology) ^{Footnote 20}. This is easily understood as CCI publishing domains are overspecialized and the publications are addressed to a small number of professionals from the conservation community ^{Footnote 21}. In both cases, the impact of CCI publications is below the global level: the median of relative citations of these publications is just under 0.70, while the median of relative impact factors just over 0.60. These scores mean that CCI publications receive about 30% fewer citations than the global norm, and are published in journals nearly 40% less visible than the global norm.

Achievement of intermediate outcome: new heritage knowledge is made available nationally and internationally

The observed reduction in research and development projects is paralleled by the reduction in the annual number of published peer- journals articles, with the linear trend line showing a reduction in the number of published journal articles by more than half, going from 19 publications in 2011-12 to 6 publications in 2015-16 ^{Footnote 22}, ^{Footnote 23}. The number of publications in professional journals has been relatively stable, fluctuating between 16 articles published in 2011-12 and 10 papers published in 2013-14 and an average of 12 papers published annually between 2010-11 and 2014-15.

Figure 5.1: number of articles published in peer-reviewed and professional journals, 2010-11 to 2015-16

Figure 5.1: number of articles published in peer-reviewed and professional journals, 2010-11 to 2015-16 – text version

Year	Total number of articles published
2010-11	17
2011-12	19
2012-13	14
2013-14	11
2014-15	15
2015-16	6

Source: CCI indicator reports

The reduction in published articles also had an impact on the ratio of number of publications by the number of CCI scientific personnel during the evaluated period. Next figure 5.2 demonstrates that CCI researchers’ time has been progressively diverted from writing articles for journals to other tasks. The reduction of research and development projects in which CCI was involved, the reduced number of articles published in journals and the reduced ratio of number of publications over number of CCI researchers all signal the same phenomenon: the reduced national and international availability of new heritage knowledge produced by CCI. Many conservation professionals retired and were replaced by less experienced staff, which had an impact on CCI’s capacity to publish.

Figure 5.2: ratio of number of publications by FTE, 2011-12 to 2015-16

Figure 5.2: ratio of number of publications by FTE, 2011-12 to 2015-16 – text version

Years	Ratio (Publications/FTE)	Target
2011-12	2.4	1
2012-13	1	1
2013-14	0.9	1
2014-15	1.4	1
2015-16	0.7	0.7

Source: CCI indicator reports

A different way of communicating preservation and conservation heritage knowledge created by CCI researchers is through presentations at Canadian and international fora. During the period under evaluation, CCI researchers presented a total of 116 presentations, 60% of which were given in Canada and 40% were rendered in international settings. While the number of presentations in Canada has varied (average 8 presentations annually) between 2010-11 and 2013-14, the number of presentations in Canada in 2014-15 and 2015-16 has increased to 19 each year. In sync with restrictions on travel for civil servants, most of the Canadian presentations were given in Ottawa, oftentimes under the auspice of CCI or other Canadian government organizations involved in conserving heritage (such as Library and Archives Canada or Parks Canada) while the international presentations were given primarily in Northeast USA settings. Both national and international presentations concerned mostly preservation services, including archaeology, as well as conservation research and sciences.

Figure 5.3: presentations by CCI researchers in Canada and internationally, 2010-11 to 2015-16

Figure 5.3: presentations by CCI researchers in Canada and internationally, 2010-11 to 2015-16 – text version

Years	Total annual presentations	Presentations in Canada	International presentations
2010-11	16	15	1
2011-12	14	4	10
2012-13	15	3	12
2013-14	18	10	8
2014-15	32	19	13
2015-16	21	19	2
Total	116	70	46

Source: CCI indicator reports

The reduction of CCI's involvement in research and development projects was remarked upon by CCI usual international and national partners in two different ways:

• many key informants commented on the limited presence of CCI on the international stage in recent years, with a few indicating that CCI appeared to be less available to participate in international research and development projects;
• A few also mentioned the decrease of CCI research and development program over the last few years and indicated that CCI is not creating new knowledge anymore.

This adaptation to the context, through the decrease in performance in research and development activities, which is at the core of CCI’s mandate (as outlined in the program theory component of the program profile – section and figure 2.1), could have an important impact on CCI’s ability to reach its intended results.

2. Knowledge dissemination

While research and development is central in CCI mandate, dissimination is essential to its mandate to ensure that the knowledge created is understood and implemented by professionals of the conservation community. CCI's aims to disseminate conservation knowledge to Canadian heritage institutions and professionals so that they have the information, data, tools, and skills to care for the collections which have been entrusted to them. CCI knowledge dissemination strategy has taken different forms and shape in the past and has evolved during the evaluation period. Some standard approaches, such as in professional training, paper publications or internships, are still part of CCI’s knowledge dissemination approach while knowledge made available through the CCI web site has taken an increasing place in the overall dissemination strategy.

Achievement of immediate outcome: heritage institutions and workers take advantage of CCI learning opportunities

The different types of learning activities proposed by CCI have been used largely by heritage institution and workers. A total of 117 regional workshops were offered by CCI between 2010-11 and 2015-16, 12 advanced workshops were organized, the number of CCI publications printed and distributed or printed and sold in and outside Canada have kept stable throughout the period while the use of CCI web resources have generally grown and internships have been stable. The main characteristics of CCI’s learning opportunites is that although the global context has been difficult for CCI, the knowledge dissemination national activities have been relatively constant during the period evaluated as the next sub-section demonstrates.

Training

CCI proposes two types of training to Canadian professionals in preservation and conservation : the regional workshops, which comprises trainings that are of a more basic nature and usually attended by heritage institution workers, which includes but not limited to new professionnals to the field of preservation and conservation, while the advanced professional development workshops intend to present, discuss and share knowledge and experience with professional conservators that are at mid-term in their carreer. Finally, regional workshops are provided on a request basis from institutions and coordinated through provincial associations of heritage organisations (also responsible for the registration of participants), as well as other clients requesting training, while advanced workshops are led by CCI.

Regional training

Between 2010-11 and 2015-16, most of CCI's regional trainings were provided on a request basis from heritage and conservation associations in Canada, large and medium sized museums, provincial and territorial governments as well as aboriginal associations or communities.

A total of 117 regional workshops were attended by an average of 16 heritage workers per workshop. As shown in figure 5.4, the number of events, as well as the average number of participants attending the workshops per year was relatively constant throughout the evaluation period. This can in part be attributed to the limit imposed by CCI on the maximum number of participants per workshop, depending on the subject. The number of participants in regional workshops annually is also dependant on the number of workshops offered. The number of regional workshops available by fiscal year is determined by CCI based on a regional formula to ensure access across the country.

Figure 5.4: average number of participants per regional training (event/workshop), 2010-11 to 2015-16

Figure 5.4: average number of participants per regional training (event/workshop), 2010-11 to 2015-16 – text version

	2010-11	2011-12	2012-13	2013-14	2014-15	2015-16
Average number of participants per workshop	14	14	19	17	17	15
Total number of workshops	22	17	23	19	21	15

Source: CCI indicator reports

Training attendees are mainly from Ontario (26%), the Atlantic region (23%) and the Prairies region (21%). Lower percentages of trainees are from the North region (12%), Bristish Columbia (10%) and Quebec (8%). As graphic (Figure 5.5) shows, the linear trend in the number of participants from most Canadian regions has increased with the exception of British Columbia and the Prairies. During this period, CCI training activities were mainly hosted by provincial museum associations, including training in French hosted by the Société québécoise des musées.

Figure 5.5: percentage of participants to CCI regional workshops by regions, 2010-11 to 2015-16

Figure 5.5: percentage of participants to CCI regional workshops by regions, 2010-11 to 2015-16 – text version

Years	Prairies	Ontario	Quebec	Atlantic	West	Northern
2010-11	30%	18%	12%	18%	10%	12%
2011-12	25%	27%	0%	24%	12%	13%
2012-13	19%	34%	7%	17%	14%	10%
2013-14	19%	26%	11%	28%	12%	4%
2014-15	25%	21%	8%	31%	7%	9%
2015-16	10%	30%	12%	22%	5%	22%

Legend:
Western region includes Yukon, BC and Alb.
Atlantic region includes PEI, NB, NS and NFLD
Prairies includes Manitoba, Saskatchewan, NWT and Nunavut

Source: PIMS standardized reports, July 2016.

Advanced professional development workshops

As mentioned earlier, this type of workshops is intended for conservators who have experience in their profession. On average (Table 5.2), two national workshops were organized annually and were attended on average by 53 participants. This average includes the higher number of participants allowed to attend the first day of each such event, which consists of lectures. Subsequent days of hands-on work typically include 20 to 25 participants. These workshops were attended by conservators from all Canadian regions, attached mostly to large and medium sizes museums, Canadian government national museums or departments/agencies as well coming from the private sector. CCI also collaborated with international organizations or with internationally recognized specialists to provide an average of one advanced professional development workshop annually in Canada during the course of the evaluation period.

Table 5.2: number of events and participants at CCI's advanced professional development workshops from 2010-11 to 2015-16

	2010-11	2011-12	2012-13	2013-14	2014-15	2015-16
Number of events	3	3	1	2	2	1
Number of participants	69	53	25	50	87	33

Source: CCI indicator reports

Internships

Internship is a different way of disseminating knowledge by providing training and preparing the future generation of conservators. During the period comprised by the evaluation, paid internships have been constant with an average of 4 paid intern but the number of unpaid interns (Curriculum) has fallen by 50% between 2012-13 and 2014-15 and thereafter due to the FTEs availability in conservation labs to supervise internships.

Table 5.3: annual number of unique paid and unpaid interns

Internships	2010-11	2011-12	2012-13	2013-14	2014-15	2015-16
Paid, unique or repeats for multi-year	6	2	4	3	3	3
Unpaid (curriculum)	8	10	15	9	7	7

Source: CCI administrative data

Publications

The number of publications ^{Footnote 24} that were printed and distributed or printed and sold in and outside Canada has fluctuated through years (Table 5.4), with a high of 2208 publications printed in 2012-13 and a low of 650 in 2014-15. Even though the type of material printed has changed during the period, the number of paper publications demonstrate that printed publications remain an important way of disseminating knowledge. The reduced number of printed publications in 2014-15 could be associated with the difficulties in accessing CCI`s online bookstore during that year ^{Footnote 25}.

Table 5.4: number of print publications distributed and sold, in and outside Canada, 2010-11 to 2015-16

	2010-11	2011-12	2012-13	2013-14	2014-15	2015-16	Annual average
Number distributed in Canada	384	216	949	608	375	566	516
Number distributed outside of Canada	405	244	395	327	86	281	290
Number sold in Canada	384	216	477	227	103	216	271
Number sold outside of Canada	405	244	387	309	86	281	285
Total	1,578	920	2208	1,471	650	1,344	1,362

Source: CCI indicator reports

Web resources

As it can be expected with the development and use of the internet, the consultation of CCI web pages has been growing regularly since the first year of the period under evaluation. This increase consultation touched both the number of pages viewed and number of unique visitors related to CCI's online learning material. As demonstrated in Figure 5.6., after an important growth in the consultation of CCI website from 2010-11 to 2013-14, the two last years of the evaluation period were marked by a pronounced diminution of the frequentation of all measured online activities. This diminution is connected to several prolonged interruptions of web services since the migration to the Government of Canada generic website, a migration that is expected to be finalized in 2016-2017. In addition, some interviewees mentioned that due to the migration, the CCI content is now more difficult to find on the government website. Although linear trend lines indicate an increase in most web consultation indicators, the realization of this trend in the future depends on the stability of the Government of Canada website, as well as access to ensure that CCI continues to disseminate knowledge.

Figure 5.6: CCI Web global and online learning traffic, 2010-11 to 2015-16

Figure 5.6: CCI Web global and online learning traffic, 2010-11 to 2015-16 – text version

	2010-2011	2011-2012	2012-2013	2013-2014	2014-2015	2015-16
Number of CCI web pages viewed	759,294	717,604	799,519	952,894	664,419	569,114
Number of CCI on-line learning page viewed	N/A	134,473	419,127	622,391	347,138	380,117
Number of unique visitors to CCI on-line learning materials	13,659	67,937	305,761	498,544	338,919	163,335
Number of CCI unique visitors	123,609	134,359	190,460	739,414	500,344	177,808
Number of on-line learning documents	N/A	1,909	1,826	2,125	639	N/A

Source: CCI indicator reports

Some of the specialized training resources available on CCI’s website were consulted tens of thousand times and CCI notes, which were added recently as updated online learning documents, have been downloaded many thousands times each years (See Tables 5.5 and 5.6 on the top three learning material viewed online and the top three CCI Notes downloaded, Appendix D).

Achievement of intermediate outcome: heritage institutions and workers improve their professional knowledge, skills and practices to preserve their collection

There is no doubt that the knowledge dissemination activities of CCI are appreciated by the Canadian heritage conservation community: both regional and advanced professional development workshops are considered of high quality and well organized by interviewees. The documentation provided at the training is considered particularly well adapted to each training. The relative stability of the number of trainings and number of participants thereof suggest a continued interest for CCI learning opportunities aimed at improving the knowledge required to preserve heritage collections.

The disappearance of the advanced professional development workshops provided to heritage institutions outside of Canada was mentioned as a loss for the international heritage conservation community.

The same applies to Web resources consultation and publications. The increase number of CCI Notes and their adoption by web users, the ascendant number of online web pages viewed, and the constant requests for printed publications demonstrate a sustained interest of the conservation community members in improving their professional knowledge.

A few interviewees made comments on the regional workshops and on the web resources. As mentioned earlier, regional workshops address basic content for generalists or non-experts in the profession. As it can be observed by the following comments, the way of the future for CCI regional workshops, as perceived by some commentators, reflect some specific needs:

• current basic trainings could be made available in a digital format and therefore would be available on a need and permanent basis;
• the regional trainings should be tailored to adjust to the needs of specific audiences; and
• there is a need for experienced heritage professionals working in conservation for trainings that fills a gap (knowledge, skills and practice) between regional workshops and the advanced professional development workshops.

3. Expert services

The third CCI pillar as well as service delivery mechanism is the expert services provided to the heritage community. As much as knowledge creation and dissemination, expert services are at the heart of CCI mandate and is for a large part the foundation for its national reputation and recognizance. Expert services are oriented toward serving the heritage community by providing three types of services^{Footnote 26}, either at CCI location or at a client location:

• Scientific services aim at providing detailed information on the materials and structure of artifacts, works of art or heritage collections to benefit the client`s stewardship of collections, and increase knowledge which supports conservation and preservation efforts.
• Conservation and treatment services aim at restoring artifacts and works of art to prevent further deterioration, aid interpretation, or re-establish culturally significant qualities. Such treatments can range from minimal stabilization to extensive restoration or reconstruction.
• Preventive conservation is mostly concerned with the prevention of deterioration, damage or loss to the heritage collections on display, in storage and in transit with the objective of identifying risks to objects and collections and to providing impartial, independent expert advice on mitigation strategies.
• CCI also responds to general information requests (GIRs), in which the majority of the requests are technical in nature and require the assistance of conservation scientists and conservators. More specifically, CCI responded to a total of 5,675 GIRs during the evaluation period for an average of 946 requests per year. The majority of requests came from requestors in Canada (76%), with 81% of requests being scientific or technical in nature.

In all cases, service requesters are provided with information that takes different forms but that provides new knowledge to institutions regarding the objects or collections analyzed, conserved or examined and ensure they are better understood and therefore better protected and conserved. These services often work together as detailed information might be needed to ensure that conservation and treatment services are relevant and correctly targeted, that the scientific analysis are based on pertinent interpretations of objects or products or that preventive conservation is informed by knowledge on treatment. Expert services are oftentimes a source to generate new knowledge, as well as tools or techniques which are translated into papers published in professional journals, into CCI publications and tools available on the CCI web site, and for new or improved training/workshops.

Achievement of immediate outcome: heritage institutions use expert services to preserve and better understand their collections

There is evidence that a variety of different heritage institutions across Canada are using a broad range of expert services offered by CCI (Table 5.7, Appendix D). Between 2010-11 and 2015, CCI completed 998 expert services projects, a yearly average of 166 services completed ^{Footnote 27}.

In the context of conservation and treatment services projects, an additional total of 256 scientific analysis services were performed internally. Therefore, a total of 1,254 expert services were completed by CCI. Scientific analysis services account for 60.3% (N=756) of all services, followed by preventive conservation services (N=268, 21.4 %) and conservation and treatment services (N=230, 18.3 %). A decrease was observed in scientific analysis services as, between 2011-12 and 2015-16, the total number of services completed diminished by 43.7% and the completed services to external clients decreased by 33.3%. A reduction of services is also remarked in Preventive Conservation services provided to external clients between 2012-13 and 2015-16 (37.0%).

Figure 5.7: number of expert services completed, 2010-11 to 2015-16

Figure 5.7: number of expert services completed, 2010-11 to 2015-16 – text version

Types of expert services	2010-11	2011-12	2012-13	2013-14	2014-15	2015-16
Conservation and treatment services	49	31	34	44	41	31
Preventive conservation	62	37	54	40	41	34
Scientific analysis	120	167	141	119	115	94
Total	231	235	229	203	197	159

Source: PIMS, standardized reports, July, 2016

A total of 328 heritage institutions have submitted 998 accepted and completed expert services requests/projects between 2010-11 and 2015-16. However, the total number of unique heritage institutions served has been decreasing over the last 6 years, from 111 institutions for which request were accepted and completed in 2010-11, to 75 by 2015-16.

Figure 5.8: number of unique heritage institutions served, 2010-11 to 2015-16

Figure 5.8: number of unique heritage institutions served, 2010-11 to 2015-16 – text version

Years	Number of unique heritage institutions
2010-11	111
2011-12	93
2012-13	101
2013-14	93
2014-15	96
2015-16	75

Source: PIMS, standardized reports, July, 2016

A total of 420 projects completed by CCI expert services were delivered to 17 heritage institutions who received ten or more expert services during the evaluated period. As Table 5.8 (Appendix D) shows, the first three institutions receiving services from CCI are for PCH, the National Gallery of Canada and Art Gallery of Ontario. CCI experts services offered to other institutionsinclude large Canadian museums, other federal government departments as well as other conservation treatment provincial institutions.

CCI offers expert services to MCPP and INDEM programs at PCH to meet legal obligations, but does not offer treatment services to these programs. Instead, CCI evaluates whether conditions in a particular museum are adequate to justify government investment. These reports are also provided to the institutions in question, and therefore serve to inform their practices.

Table 5.9 presents the type and percentage of institutions that received expert services from 2010-11 to 2015-16, based on the 4 most frequently used services.^{Footnote 28} These services were delivered by CCI to large Canadian public museums with a budget over 1M$ (51.6%), followed by medium-sized museums with budget between 100,000$ and 999,999$ (10.3%), and small museums with budgets under 100,000$ (4.3%).

Other recipients of CCI experts services over the last 6 years are FPT governments (14.7 %,), the general public, including the private sector – Canadian or foreign (8.1%), and for PCH (4.4%).

Table 5.9: percentage of type of institutions served by CCI expert services, 2010-11 to 2015-16

Organization types	Number	Percentage
Large Canadian public museums - budget >$1,000,000	326	51.6%
FPT Governments	93	14.7%
Medium-sized Canadian public museums - budget $100,000 to $999,999	65	10.3%
General public, private sector or canadian public	51	8.1%
Department of Canadian Heritage and Portfolio Agencies	28	4.4%
Small Canadian public museums - budget <$100,000	27	4.3%
Educational/training programs	19	3.0%
Organizations - religious communities/societies in Canada	11	1.7%
Organizations - aboriginal communities/societies in Canada	10	1.6%
Heritage and conservation associations - Canada	2	0.3%
Total	632	100.0%

Source: PIMS administrative data, July, 2016.

The majority of services provided by CCI are offered in Ontario (42.8%) and Quebec (39.3%) which represents the majority of all services provided by CCI, followed by Western region (10.3%), Prairies and Northern Region (PNR) (4.8%), Atlantic (3.0%) (Table 5.10).

Table 5.10: regional distribution of services completed by CCI, 2010-11 to 2015-16

Region	Number	Percentage
Ontario	342	42.8%
Quebec	314	39.3%
Western region	82	10.3%
Prairies and Northern Region	38	4.8%
Atlantic	24	3.0%
Total	800	100.0%

Source: PIMS administrative data, July, 2016

Heritage institutions sent 1292 objects or products to CCI laboratories for scientific analysis in the context of the 500 requests/projects serviced. During the period evaluated, the number of objects/products sent to CCI for scientific analysis has increased (Figure 5.9).

Figure 5.9: total number of objects analysed by CCI, 2010-11 to 2015-16*

Figure 5.9: total number of objects analysed by CCI, 2010-11 to 2015-16* - text version

Years	Total number of objects
2010-11	104
2011-12	119
2012-13	181
2013-14	135
2014-15	385
2015-16	368

*Potential methodology change starting in 13-14
Source: CCI indicator reports

A total of 19% of expert services provided are treatment services (N=192) which represents an average of 32 conservation treatment services per year (includes conservation treatment services at CCI, at client site and archeology). A total of 1,568 objects were treated and restored over the last 6 years by CCI. A total of 29% of objects (N=457) were archeological object types. These figures do not include the number of objects and collections that were treated as a result of CCI providing the technical report/protocol to the institutions. It is difficult to determine to what extent objects and collections are treated by institutions as a result of CCI providing their expertise through technical reports/protocols.

CCI indicator reports showed that the total number of objects treated and restored by CCI has steadily increased over the last 6 years, from approximately 135 in 2010-12 to more than 400 in recent years (2014-16).^{Footnote 29} The treatment and restoration of non-archeological objects are primarily responsible for the increase (furniture, paper, textile, etc.).

In addition to objects that were restored by CCI experts, some of the expert services projects also aimed at collections. A total of 308 projects were dedicated at supporting heritage institutions in conserving collections (Table 5.11) with the largest part of the projects related to providing facilities advisory services (47.7%) and to loaning equipment needed to support the conservation of collections (33.4%).

Table 5.11: number of collections serviced by CCI expert services (completed services), 2010-11 to 2015-16

Expert service types	2010-11	2011-12	2012-13	2013-14	2014-15	2015-16	Total
Archaeological field service	1	0	0	1	0	0	2
Collection survey / risk assessment survey	1	2	1	3	1	0	8
Emergency response	3	0	1	3	1	2	10
Equipment loan	20	19	17	15	21	11	103
Facilities advisory service	38	16	35	19	18	21	147
Scientific service at CCI - Object analysis and examination	2	4	0	2	0	0	8
Scientific service at CCI - product and material evaluation	3	6	7	3	10	1	30
Grand total	68	49	69	53	54	39	308

Source: PIMs standardized reports, July 2016.

Case studies and key informant interviews indicated that heritage institution representatives and professionals:

• recognized CCI unique expertise in conservation sciences and treatment,
• mentioned that the services met their needs,
• indicated being very satisfied with the high level of quality and usefulness of expert services provided by CCI, many interviewees tracing it to earlier times, and
• emphasized that the work provided by CCI is important to their institutions.

In addition, heritage institutions have shown a high level of satisfaction towards CCI expert services in CCI's annual surveys. The surveys indicate satisfaction rates fluctuating between 95% and 99%, thus achieving CCI’s target of 90% (revised from 95% to 90% in 2013-14). Given the limitations with the methodology used for the client satisfaction survey ^{Footnote 30}, it is suggested CCI revises its current approach to document the processes by which client satisfaction surveys are administered in order to increase representation and usefulness of information.

Nevertheless, while most interviewed heritage institutions and professionals agreed that preventive conservation and scientific analysis services are provided in a timely fashion, many raised concerns regarding the timeliness of conservation treatment services (case studies and key informant interviews). While satisfied with the services, many key informants indicated that the treatment period can be lengthy, with many pointing toward capacity issues such as resources availability and lab closures as potential sources of service delays. Results from case studies show that in spite of these concerns, interviewees indicated that objects were treated and returned in time for special events organized by institutions.

Achievement of intermediate outcome: heritage cultural objects and collections are preserved and better understood

Results from cases studies show that heritage institutions representatives, as well as professionals mentioned that cultural objects that were analyzed, treated and/or restored as well as collections that were protected by CCI, are usually rare objects that have a significant heritage importance and are meaningful for the understanding of important events and persons. They are part of the Canadian heritage and bear an important significance for Aboriginal peoples, Canadian communities, and, more largely, Canadian history. Through the support of CCI experts, they are conserved and their heritage value is shared for the understanding, appreciation and enjoyment of present and future generation.

Results from these lines of evidence also show that all institutions consulted for the evaluation agreed that CCI’s reports, treatment protocols and recommendations:

• were of excellent quality and useful;
• were sometimes used as evidence necessary to receive approval for funding to make required changes to installations with the purpose to better preserve and conserve their objects and collections ^{Footnote 31};
• have given the opportunity to professionals, through better understanding of the objects treated or collections conserved, to develop interpretations that reflect adequately the nature of the objects and their historical significance.
• allowed them to better understand preservation techniques of objects and collections and related conservation practices and
• were very useful to restore similar objects in their collections.

4. Achievement of ultimate outcomes

Ultimate outcome: heritage institutions and workers are preserving their collections

Results from case studies and case studies indicate that CCI is providing the heritage institutions with high quality and useful tools to support heritage institutions and workers in preserving objects that are part of collections. Results from these lines of evidence also show that all activities and related products of CCI allows for better long-term preservation and care of the objects/collections to ensure that objects of historical significance are available for future generations:

• products stemming from experts services (treatment protocols, expert service reports and advice) provided to heritage institutions;
• regional and advanced professional development trainings;
• knowledge disseminated through notes, paper publications and web resources;
• knowledge created through research and development.

Ultimate outcome: CCI services enable heritage institutions to reach their goal of preservation and accessibility.

Results from case studies show that most institutions indicated that CCI’s services helped them better understand and interpret the objects that were treated and that, consequently, they were able to provide more accurate and in-depth information about objects and collections to communities and Canadians that visit heritage institutions.

The increase information and understanding about the historical importance and significance of the objects allowed institutions:

• to better showcase the contribution of aboriginals in Canadian History, like the role of the Six Nations during the War of 1812;
• to provide more relevant information during guided tours of their heritage institution and thus increasing the cultural ‘experience’ of visitors.
• to allow museum visitors to have a better appreciation of the heritage object or art that is displayed in front of them.

Most objects treated and restored by CCI:

• were made accessible and displayed for community and public viewing in permanent or temporary exhibits when returned to their respective institutions
• were displayed as part of celebrations and ceremonies, like the bicentennial anniversary of the War of 1812;
• were made available to other museums/galleries for public viewing through institutional loan programs; or
• will be displayed as part of planned future exhibitions.

Finally, case studies also show that institutions modified the organisation of their objects/collections/exhibits displayed to increase the cultural experience of visitors and/or to better preserve their objects/collections as a result of the services provided by CCI.

5.2. Core issue 5: demonstration of efficiency and

The evaluation has commissioned an independent consultant to review the appropriateness of the current Special Operating Agency (SOA) status of CCI, considering the changing context the Institute went through during the evaluation period. The CCI’s SOA status was delivered by the Treasury Board in 1992 and was reconfirmed in 2010. It gives CCI the authority and responsibility to achieve results for Canadians effectively and efficiently, in the area of heritage conservation, as well as the flexibility required in the specialized and complex field. As an SOA, the Institute is committed to a business-oriented corporate culture and has a primary objective of service delivery.

One characteristic of the SOA status is that it usually comes with greater flexibility to meet the higher performance expectation in achieving objectives. For CCI, this means having flexibility in building CCI expertise, modernizing and diversifying opportunities for professional development and enhancing operational efficiencies in a highly specialized scientific environment with a service-orientation. This flexibility is seen as necessary and can be observed in other Canada government organizations operating under a SOA status (Table 5.12, Appendix D) as well as in a few comparable international organizations ^{Footnote 32} who all have control over their important corporate functions.

Currently, following a centralization of all corporate services at PCH, CCI relies on the Department to provide the services it needs. However, it could be argued that CCI is currently hampered in being an organization that is service oriented and will find it increasingly difficult to meet the SOA’s intention of being service oriented without having the necessary flexibilities

To what extent is CCI delivered efficiently?

As alluded to in the context section of this report, CCI undertook a reduction of 28.1% of its positions to reduce overall costs due to cuts in its budget and following leaves of professionals that could not be easily replaced. As shown in Table 5.13, 54.5% of administrative positions were lost or transferred to central services and 20.4% of positions related to all types of professional services provided by CCI ^{Footnote 33}. Only positions related to knowledge sharing remained relatively constant throughout the evaluation period.

Table 5.13: staff reductions at CCI, 2010-11 to 2015-16

CCI staff	2010-11	2011-12	2012-13	2013-14	2014-15	2015-16
Administrative support	22	20	18	11	9	10
Services*	54	51	50	48	43	43
Knowledge sharing -	13	12	12	11	12	13
Total	89	83	80	70	64	66

* Aggregation of cost centers that are directly connected to services to clients (not administrative in nature)

Source: PCH administrative data, March, 2017.

To ensure continuity of services in regard of the important changes that CCI underwent during the period comprised by the evaluation and the nature of the work done by CCI employees, CCI adapted the rendering of its services to maintain a level of performance that ensured the overall quality of the tasks required. To do so, CCI used different approaches:

• Reduction of both research and development activities and expert services;
• Reduction of performance or client services targets (annual number of publication per FTEs, annual level of general satisfaction of expert services clients, annual number of heritage institutions served) ^{Footnote 34}.

An examination of the number of days it took CCI to complete projects in the three expert services aimed specifically at the Canadian heritage community and over which CCI has control ^{Footnote 35} shows that its services went through an extended period of difficulties in delivering its services, with an average of 80 days required to complete all services in 2011-12 that increased to an average of 194 days in 2013-14, before falling to an average of 133 days in 2014-15. The conservation treatment services were impacted by this increase, with the number of days required to complete projects being multiplied by 1.7 times in 2013-14 and 1.3 times in 2014-15 compared to 2011-12. The number of days to complete the two other expert services (Preventive conservation services and Scientific Analysis services) for the same years were respectively multiplied by 2.2 times and 1.7 times for preventive conservation services and 1.9 times and 1.8 times for scientific analysis services.

Table 5.14: number of days required to complete projects, 2011-12 to 2014-15 ^{Footnote 36}

Types of expert services	2011-12	2012-13	2013-14	2014-15
Conservation treatment services	222.1	287.1	365.8	298.3
Preventive conservation services	107.5	145.1	233.3	180.3
Scientific analysis services	52.8	107.0	102.4	95.8
Average number of days for all expert services projects	79.9	137.2	194.4	133.3

Source: PIMS administrative data, January, 2017

The evaluation also looked at the data available concerning the forecasted and actual time needed to complete expert services projects (Table 5.15). In 2011-12, on average, projects were completed 9.3 days later than forecasted. During the following years, CCI had difficulties in accurately forecasting the time required to complete projects: the difference between forecasted and actual time to complete projects shows that expert services projects planning was off by an average of 73.7 days per project in 2013-14 and 45.4 days in 2014-15. Planning of both conservation treatment and preventive conservation services projects was affected, showing the larger differences in comparison with 2010-11 in 2013-14.

Table 5.15: average difference in days between forecasted and actual number of days required to complete projects, by types of service, 2011-12 to 2014-15^{Footnote 37}

Types of expert services	2011-12	2012-13	2013-14	2014-15
Conservation treatment services	9.7	-29.6	-155.8	-25.7
Preventive conservation services	-28.1	-102.0	-164.0	-94.2
Scientific analysis services	-8.9	-41.1	-15.2	-40.4
Total	-9.3	-55.6	-73.7	-45.4

Source: PIMS administrative data, January, 2017

These results must be compounded by the reduction (10.7%) in the number of projects completed (Figure 5.7) between 2010-11 and 2014-15) and could be extrapolated to 2015-16. By reducing the number of projects completed, the number of days required to complete as well as the average differences in days between the forecasted and actual number of days fell. It is expected that the 20.4% decrease in the number of projects observed between 2011-15 and 2015-16 had an even greater impact in both measures.

Considering that the service requestors were told of the forecasted date for the completion of their projects, it is not surprising that some interviewees mentioned that, independently of their full appreciation for the tasks completed, it sometimes took much longer than they expected. A few interviewees also mentioned that requests for updates and follow-ups as well as responses to enquiries were problematic at times. CCI should review the methodology used to establish forecasted project completion dates, as well as allow the client to indicate in the client agreements often the client would like to receive updates (quarterly, semi-annually, etc.).

Evidence indicates that the selection process used by CCI to retain applications for expert services is transparent. Nevertheless, the transparency of the process could be improved by better enhancing the visibility on the CCI website on what basis applications will be assessed. In addition, the evaluation found that there is no evidence indicating that applicants whose requests for object analysis were denied got a letter to confirm and explain the refusal. CCI interviewees however mentioned that these applicants were contacted by phone or email and were therefore informed of CCI decisions.

CCI human resources is at the center of the services it provides. It is expected that an important percentage (43%) of its scientific and conservation specialists could retire in the coming years. However, the recruitment of bilingual professionals of equal caliber is challenging and time-consuming and requires specialized staffing actions that PCH Human Resources is not used to do. In addition, replacements coming to CCI will have a generalist knowledge of the conservation profession and not the knowledge and experience of those they replace. They will not have the same level of efficiency in completing their tasks until they gain significant experience in their specific conservation field. The level of effort required from the current employees should continue to be assessed and workload adjusted until CCI has been able to complete its staffing activities and has used to its best available new technologies.

Finally, CCI has taken initiatives to improve its efficiency, including by planning on digitalizing some of its in-person training activities with the objective of reinvesting the time spent by employees on trainings on other tasks. However, this specific initiative is impeded by the limitations related to posting on the Government of Canada website (Canada.ca) and the different policies that govern this type of communication activities.

To what extent does CCI duplicate, overlap or complement other programs in Canada?

The evaluation has compared the mandates and missions of a few Canadian institutions involved in heritage conservation, namely Parks Canada, Library and Archives Canada as well as the Centre de conservation du Québec, a provincial institutions dedicated to conservation treatment. ^{Footnote 38}

The results of the comparison indicates that no duplication or overlapping were present when considering CCI mandate and activities. CCI distinguishes itself by the role it imparts to conservation research and development in the fulfilment of its mandate and the support it brings to heritage institutions in Canada by offering a comprehensive suite of expert services that is not limited to conservation treatment.

5.3. Other evaluation questions

Performance measurement

The current CCI PMERS is mostly adequate to document its results in view of evaluation. Most results indicators are actively monitored and reported upon. Nevertheless, some limitations in the performance measurement strategy have been identified:

• CCI has not defined target numbers for its main services. In the absence of such targets, it is difficult to clearly assess its efficiency and the impact of the context on it;
• The methodology of some performance measures need to be reviewed to ensure that they accurately reflect the activities and results of CCI, especially indicators important to measure evolution over time (number of expert services transactions, number of research and development projects and number of partners);
• The methodology used for CCI’s client surveys must be reviewed, completed and implemented (sampling of clients who receive expert services, measurement of behavior changes (practices) as a result of CCI’s training);
• Some data on service standards expected to be posted on CCI’s website is not captured.

In addition to difficulties encountered during the evaluation with getting data in a format that can be properly analyzed, all lines of evidence as well as a review of CCI’s system used to track CCI projects show that performance information is not and cannot be well integrated into CCI’s decision-making process. Improvements must be made to CCI’s data retrieval and reporting system to ensure the relevance, quality, integrity, reliability and validity of the data and the proper activity monitoring and reporting.

Previous evaluation recommendations

The recommendations stemming from the 2011 evaluation were as follow:

CCI should develop and implement a strategic business plan to guide its service delivery, and to do so in consultation and communication with heritage institutions and associations in Canada.

Important initiatives in the plan should include:

• Exploring an integrated approach with PCH Heritage programs to serve mutual clients;
• Identifying issues and challenges facing heritage institutions including the types of research, training and knowledge dissemination needed to support institutions;
• A long-term research plan that is responsive to and aligns with the needs of the Canadian heritage institutions;
• Adapting and diversifying efforts in training, information dissemination and professional development to respond to the needs of heritage professionals; and
• Ensuring continuity of excellence and preparing for tomorrow’s challenges by developing a succession and talent management approach for conservators and conservation scientists. This approach should also take into account the needs for recruiting bilingual experts.

All requirements concerning the recommendations were implemented, culminating with the development and approval of three 2015-2020 Plans: CCI Strategic Plan, CCI Professional Development Plan and CCI Research Strategy. However, the program theory of CCI, its complexity and the interdependency of its delivery mechanisms in producing results, the limitations related to the operational context and the findings of the evaluation concerning the reduced research and development activities and expert services outputs all cast a shadow on the potential for a full implementation of the Plan. The assessment of their implementation is out of the scope of the current evaluation and a formative evaluation, that could be conducted in 2018-19, might be an efficient tool to measure the level of their implementation and their impacts on the overall situation of the Institute.

Official languages

CCI has made efforts during the period under evaluation to fully meet its commitments to the official language requirements. CCI

• has signed a partnership with the CCQ to ensure that conservation training is available in French in regions with significant French speaking minorities;
• supports the language development of its employees;
• has initiated (2014-15) the collection of information related to training services provided to official language minority communities professionals and organizations and should find ways of identifying expert services provided to organizations deserving language minority communities; and
• makes sure that all services, information and documents are available in both official languages.

However, some French speaking interviewees have mentioned that, although trainings and their related materials are presented in French, the bibliographic references listed in the material are usually exclusively in English, limiting the possibility of exploring the training subjects more in depth in their official language. When available, CCI should provide bibliographic references to training materials produced in French and therefore extend the national and international sources of knowledge used to develop new knowledge and to train heritage professionals.

Gender based analysis plus

The analysis of gender equality at CCI is more or less relevant for the evaluation of this program, considering that CCI’s clients are mostly institutions and that the majority of CCI’s employees are females. CCI could nevertheless ensure that the regional and advanced trainings registration database includes data related to the gender of participants. Since clients that are often provincial associations are responsible to host the training events, CCI is not responsible for the registration of participants at the regional workshops.

There is no doubt that CCI has taken into consideration the services provided to aboriginal, multicultural as well as official language minority communities and has reported on its performance. Nevertheless, the capacity of CCI to report accurately has been impeded by the considerations already mentioned previously in the Performance measurement section. However, CCI has revised its data collection tools in 2015-16, mainly around services to aboriginal communities to improve its ability to clearly report on the services provided to these communities.

6. Conclusions and recommendations

6.1. Conclusions

CCI is an important organization in the larger context of heritage institutions in Canada. Its unique research and development, preventive conservation and conservation treatment services support a large ensemble of Canadian heritage community institutions. The services are responding to the needs of the heritage community, are deemed as of excellent quality, are recognized as useful to maintain and conserve heritage and as being an important value added to heritage institutions of all sizes and parts of Canada, heritage institutions that often do not have the resources to conduct by themselves conservation work of that nature.

However, a confluence of factors is conducive to challenging the ability of CCI to successfully continue serving the heritage community to the same extent as a science institute. The context into which CCI has evolved during the period covered by the evaluation and the necessity to adapt to that new context has had an increasing impact on the achievement of its results and its long term impacts are clearly observable: limited international collaborations, reduced initiation of and contribution to research and development projects, important reduction in peer reviewed publications annual numbers, reduced expert services activities.  Considering the program theory on which CCI activities is founded, the expectations in terms of achievement of its results in the future have to be reduced.

Notwithstanding the reduction of its program in recent years, CCI remains a relatively nimble tool to support heritage institutions in Canada as well as government priorities (as exemplified by the recent work related to the War 1812 and the Road to 2017). Other opportunities to serve Canadians could come in the contexts of the reconciliation with Indigenous peoples, Open Government, climate changes or international cooperation for the protection of cultural heritage.

6.2. Recommendations and management response

Action plan

Action item	Deliverable(s)	Timeline	Program official responsible
1.1 Identify specific administrative governance structures and processes which negatively affect CCI’s capacity to fulfill its mandate.	Analysis of opportunities to maximize administrative flexibilities, with options for consideration.	September 30, 2017	CCI DG
1.2 Enter into discussions with the relevant authorities in order to identify preferred options.	Preferred options to be presented to the DM for approval.	December 31, 2017	CCI DG
1.3 Develop plan and implement measures approved by the Deputy Minister.	Implementation plan and implementation of identified transfers in administrative processes and related resources.	December 31, 2018	CCI DG

Full implementation date
December 31, 2018

Action plan

Action item	Deliverable(s)	Timeline	Program official responsible
2.1 Gather data collection, reporting and usability requirements to enhance the capacity of CCI’s information management system to support business intelligence and reporting activities.	Business Requirements Document	October 30, 2017	CCI DG
2.2 Develop a plan for the Preservation Information Management System’s (PIMS) next phase of development.	Project Agreement with the Chief Information Officer Branch	March 31, 2018	CCI DG
2.3 Implement measures specifically addressing identified data integrity gaps and reporting capacity.	Stage one of PIMS enhancements, as identified in the Project Agreement, is completed.	September 30, 2018	CCI DG
2.4 Fully implement the Preservation Information Management System’s (PIMS) next phase of development.	Remaining PIMS enhancements, as identified in the Project Agreement, are completed.	March 31, 2022, contingent upon CIOB’s capacity to complete the deliverables by this date.	CCI DG

Full implementation date
March 31, 2022

Appendix A: TBS core evaluation issues

The Government of Canada requires that evaluations support the following:

• Accountability, through public reporting on results;
• Expenditure management;
• Management for results; and
• Policy and program improvement.

The core evaluation issues used to guide this evaluation are:

Relevance

Issues	Relevances
Issue #1: continued need for program	Assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians
Issue #2: alignment with government priorities	Assessment of the linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes
Issue #3: alignment with federal roles and responsibilities	Assessment of the role and responsibilities for the federal government in delivering the program

Performance (effectiveness, efficiency and economy)

Issues	Performances
Issue #4: achievement of expected outcomes	Assessment of progress toward expected outcomes (incl. immediate, intermediate and ultimate outcomes) with reference to performance targets and program reach, program design, including the linkage and contribution of outputs to outcomes
Issue #5: demonstration of efficiency and economy	Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes

Appendix B: CCI logic model

Inputs	Activities	Outputs	Immediate outcomes	Intermediate outcomes	Ultimate outcome	Departmental strategic outcome
Human resource Financial resources Equipment Facilities	Research and development Knowledge dissemination Expert services	Articles published in peer-reviewed publications; presentations to peers Publications and Web resources Training events Conservation treatments Scientific services Facilities assessments Archaeological services	New knowledge in heritage preservation and conservation is created. Heritage institutions and workers take advantage in CCI learning opportunities. Heritage institutions use expert services to preserve and better understand their collections.	New knowledge in heritage preservation and conservation is disseminated nationally and internationally. Heritage institutions and workers improve their professional knowledge, skills and practices to preserve their collection. Heritage cultural objects are preserved and better understood.	Heritage institutions and workers are preserving their collections. CCI services enable heritage institutions to reach their goals of preservation and accessibility.	Canadian artistic expressions and cultural content are created and accessible at home and abroad.

Appendix C: evaluation framework

The proposed evaluation issues and questions, their associated indicators and methods to be used for obtaining required information is presented in the table below.

Relevance
Issue #1: continued need for program - assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians

Questions	Indicators for each question	Data sources for each indicator	Methods of data collection
How does CCI serve the needs of Canadians?	Continued interest in heritage objects and collections. Significance of history and heritage to Canadians.	Articles, reports, surveys and publications	Literature Review
Is CCI relevant for heritage institutions and workers in preserving Canada’s heritage collections?	Evidence and views of key informants on the extent to which there is a demonstrable need to be addressed regarding preservation and conservation in Canada. Results of stakeholder consultations undertaken by CCI.	Officials, stakeholders and clients Articles, reports, surveys and publications CCI documents and data	Document review Key informant interviews
Is CCI responsive to the needs of Canada’s heritage institutions?	Trends in CCI demand (acceptance and refusal) and work volume/outputs over time. Results of stakeholder consultations undertaken by CCI. Evidence and views of key informants on the extent to which CCI is responsive to stakeholders needs.	Officials, stakeholders and clients PIMS database Corporate business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Document and database review Key informant interviews Case studies

Relevance
Issue #2: alignment with government priorities - assessment of the linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes

Questions	Indicators for each question	Data sources for each indicator	Methods of data collection
To what extent are CCI objectives aligned with governmental priorities and the Department’s strategic objectives?	Extent to which CCI’s mandate and outcomes are aligned with: the current priorities of the federal government. the strategic outcomes of PCH. Evidence that CCI objectives and outcomes are aligned with governmental and Departmental strategic outcomes.	Government of Canada and PCH publications PCH priorities Link with other PCH programs	Document and literature review MAP evaluation report MCP evaluation report INDEM evaluation report

Relevance
Issue #3: alignment with federal roles and responsibilities - assessment of the role and responsibilities of the federal government in delivering the program

Questions	Indicators for each question	Data sources for each indicator	Methods of data collection
To what extent is CCI aligned with the federal government’s roles and responsibilities?	Extent to which CCI is harmonized with the roles and responsibilities of the federal government.	Federal budgets Speech from the Throne RPP and DPR Law and policies International conventions (UNESCO)	Document and literature review

Performance (effectiveness, efficiency and economy)
Issue #4: achievement of expected outcomes - assessment of progress toward expected outcomes
Immediate outcomes

Questions	Indicators for each question	Data sources for each indicator	Methods of data collection
To what extent did CCI reach the immediate outcome that heritage institutions and workers take advantage of CCI learning opportunities?	Outputs summary, e.g., number of institutions hosting CCI learning programs, number of users of CCI learning programs and materials and the type of institutions to which they belong, number of online page views for learning materials, number of publications mailed/sold. Number of heritage workers trained vs potential trainees. Adequacy between trainee’s expertise and training level. Level of satisfaction, usefulness and importance of knowledge training services and products. Extent to which CCI’s interventions are aligned with the learning needs of heritage institutions and professionals.	PCH officials, stakeholders and clients PIMS database Government of Canada survey of heritage institutions (2011 and 2015) Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Literature review Document review and database review Key informant interviews Case studies
To what extent did CCI reach the immediate outcome of creation of new heritage preservation and conservation knowledge?	New knowledge created by field of expertise. Connections between training, fields of expertise and research and development. Ratio of CCI articles published in Canadian and international professional and peer-reviewed journals to number of FTE conservation scientists.	PCH officials, stakeholders and clients PIMS database Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Literature review Document review and database review Key informant interviews Case studies
To what extent did CCI reach the immediate outcome of heritage institutions using expert services to preserve and better understand their collections	Outputs summary, e.g., number of requests, accepted and rejected. number of expert service transactions. number of institutions served. Type and number of heritage institutions that benefited from Canadian Conservation Institute expert services including preventive conservation services provided in association with the MAP, MCPP, INDEM and CCSF. Transparency of the selection process. Level of satisfaction, usefulness and importance of scientific services, treatments and preventive conservation services. Extent to which CCI’s interventions are aligned with the needs of heritage institutions and professionals. Numbers of objects preserved by CCI. Numbers of objects analyzed by CCI.	PCH officials, stakeholders and clients PIMS database Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Literature review Document and database review Key informant interviews Case studies

Intermediate outcomes

Questions	Indicators for each question	Data sources for each indicator	Methods of data collection
To what extent did CCI reach its intermediate outcome that heritage institutions and workers improve their professional knowledge, skills and practices to preserve their collection?	Percentage of participants who report an improvement in professional knowledge, skills and practices. Adequacy between training and heritage institutions/workers’ needs. Heritage institutions and heritage workers have changed their practices. Heritage workers use acquired professional knowledge and / or skills in their work.	PCH officials, stakeholders and clients PIMS database Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Literature review Document and database review Key informant interviews Case studies
To what extent did CCI reach its intermediate outcome that new heritage knowledge is made available (and used Footnote 39) nationally and internationally?	Level of awareness among international curators and related scientific community of CCI research outputs. Quality of scientific research published in journals.	PCH officials, stakeholders and clients PIMS database Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data Science-metrix	Literature review Document and database review Key informant interviews Case studies
To what extent did CCI reach its intermediate outcome of heritage cultural objects and collections are preserved and better understood?	Heritage importance of objects and collections preserved. Perceived usefulness of reports provided to clients. Perceived quality of reports provided to clients. Results of CCI’s intervention on objects and collections in heritage.	PCH officials, stakeholders and clients PIMS database Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Literature review Document and database review Key informant interviews Case studies

Ultimate outcomes

Questions	Indicators for each question	Data sources for ach indicator	Methods of data collection
To what extent did CCI reach its ultimate outcome that heritage institutions and workers are preserving their collections?	Heritage institutions and workers consider themselves better able to oversee and preserve objects and collections. Extent to which objects and collections are preserved.	PCH officials, stakeholders and clients PIMS database Articles, reports, surveys and publications CCI documents and data	Literature review Document and database review Key informant interviews Case studies
Are CCI services enabling heritage institutions to reach their goals of preservation and accessibility?	Heritage collections and objects, or related knowledge, are shared with Canadians through different means (e.g., events, permanent exhibitions, traveling exhibitions, made digitally available, etc.) Heritage collections and objects generate interest among Canadians.	PCH officials, stakeholders and clients PIMS database CCI documents and data Social media Media	Literature review Document and database review Key informant interviews Case studies

Performance (effectiveness, efficiency and economy)
Issue #5: demonstration of efficiency and economy - assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes

Questions	Indicators for each question	Data sources for each indicator	Methods of data collection
Is the current organizational model (SOA) still appropriate for CCI?	Appropriateness of the current SOA organizational model. Current TBS assessment of the SOA mechanism. Extent to which CCI is appropriately structured to deliver its mandate. Extent to which the organization is service oriented. Comparison with similar international institutions.	PCH officials TBS documents CCI documents and data	Document review and analysis Key informant interviews Literature review
To what extent is CCI delivered efficiently?	Planned versus actual expenditures by year Year to year comparison of administrative costs to total budget. Evidence and view of key internal informants regarding the efficiency (including design and delivery, equipment and infrastructure) of CCI in producing outputs and achieving its outcomes. Average cost by conservation business activity. (Program total costs (salaries, operations and building management) / total number of internal processes directly linked to conservation business activities). Adequacy and relevance of performance efficiency indicator.	PCH officials PIMS database Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents Financial information	Document and database review Key informant interviews
Were adequate management and administrative practices in place for effective service delivery?	Level of stakeholders and clients satisfaction with CCI services (eligibility criteria, process). CCI’s administrative practices are supporting service requirements.	Stakeholders and clients Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Document review and analysis Key informant interviews Case studies
To what extent does CCI duplicate, overlap or complement other programs in Canada?	Extent to which CCI duplicates, overlaps or complements other programs at the federal provincial/territorial level. (LAC, Park Canada, Canada School of Public Service post-secondary institutions) and stakeholders (MAP, INDEM, MCPP) Position of CCI in the conservation/preservation sector. Evidence of duplication, overlap or complementarity in programming with partners	Officials, stakeholders and clients Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Document review and database review Key informant interviews
What is the impact of the revenue generating services on CCI services that are offered for free?	Annual selection process for expert services. Financial management of CCI’s revenue. Balance between free and revenue generating services.	CCI documents Financial information	Document review and analysis

Performance (effectiveness, efficiency and economy)
Other issues

Questions	Indicators for each question	Data sources for each indicator	Methods of data collection
Is the current performance measurement framework effective at capturing the results of CCI?	Extent to which performance monitoring and measurement activities were effective at capturing the results of CCI. Use of performance data and information in decision making. Usefulness of performance data and information for program design and/delivery improvements. Extent to which performance measurement data collected was useful to the evaluation.	Officials PIMS database Integrated business plans CCI annual review Articles, reports, surveys and publications CCI documents and data	Literature review Document and database review Key informant interviews Case studies
Were all official language requirements met by CCI?	Availability of training in both official languages. Extent to which official language requirements were met for CCI.	CCI documents and data	Document and database review and analysis Surveys Case studies
Has GBA+ Footnote 40 been considered?	Extent to which the GBA+ has been taken into consideration at CCI. Has baseline information been captured to allow an assessment the impact of CCI on diverse groups of women and men. Has the collected data been disaggregated by sex/gender and other diverse factors. Differences (in any) on how diverse groups access and/or experience the services offered by CCI Equitable access to the services by diverse groups of women and men. Training is available to those offering professional training to ensure that they are sensitive to the needs of diverse groups; The perspective of diverse groups have been considered in the development of CCI’ operational and/or strategic plans, as well as the service delivery model.	Officials PIMS database CCI documents and data	Document review and database review and analysis Case studies
Are there any unexpected results identified by the evaluation?	Trainees are able to inform and train their colleagues. Any other expected results.	PCH officials, stakeholders and clients CCI documents and data	Interviews Case studies

Appendix D: additional detailed tables and figures

Table 5.5: top three learning material viewed online, 2010-11 to 2015-16

2010-11	2011-12	2012-13	2013-14	2014-15	2015-16
Ten Agents of Deterioration (7,932)	Caring for objects (4,044)	Light, Ultraviolet and Infrared  (18,629)	Light, Ultraviolet and Infrared (33,042)	Recognizing metals and their corrosion products (27,697)	CCI Notes (125,043)
Environmental Guidelines for Museums — Temperature and Relative Humidity (RH)  (5,348)	Caring for collections (3,335)	Physical Forces (9,055)	Physical Forces (11,954)	Light, Ultraviolet and Infrared  (12,038)	Care of objects and collections (108,236)
Combatting Pests of Cultural Property (2,776)	CDs and their Longevity-FAQ  (2,847)	Light Damage Calculator  (5,750)	Incorrect Relative Humidity (1,632)	Light Damage Calculator (3,817)	Agents of deterioration (62,334)

Source: CCI indicator report

Table 5.6: top three CCI notes accessed online, 2010-11 to 2015-16

2010-11	2011-12	2012-13	2013-14	2014-15	2015-16
How to Care for: Paper Documents and Newspaper Clippings (1,351)	Longevity of Recordable CDs and DVDs – CCI Notes 19/1 (1,715)	Care of Ivory, Bone, Horn, and Antler - CCI Notes 6/1 (2,688)	Care of Objects Made from Rubber and Plastic – CCI Notes 15/1 (13,449)	Care of Objects Made from Rubber and Plastic – CCI Notes 15/1 (5,891)	Care and Cleaning of Unfinished Wood - CCI Notes 7/1 (3,590)
How to Care for: Books (1,156)	The Cleaning, Polishing and Protective Waxing of Brass and Copper – CCI Notes 9/3 (2,030)	Care of Objects Made from Rubber and Plastic – CCI Notes 15/1 (2,557)	Removing Mould from Leather – CCI Notes 8/1 (10,525)	Care and Cleaning of Unfinished Wood – CCI Notes 7/1 (4,186)	Care of Objects Made from Rubber and Plastic – CCI Notes 15/1 (3,208)
How to Care for: Brass, Copper, and Bronze (1,083)	Tannic Acid Treatment – CCI Notes 9/5 (1,592)	Silver — Care and Tarnish Removal – CCI Notes 9/7 (2,529)	Care of Ivory, Bone, Horn, and Antler - CCI Notes 6/1 (9,754)	Tannic Acid Coating for Rusted Iron Artifacts – CCI Notes 9/5 (3,103)	3. Removing Mould from Leather - CCI Notes 8/1 (1,972)

Source: CCI indicator report

Table 5.7: number of completed expert services projects, by type of services, 2010-11 to 2015-16

Types of expert services	2010-11	2011-12	2012-13	2013-14	2014-15	2015-16	Total
Conservation and treatment services	49	31	34	44	41	31	230
Percentage	26.2%	18.2%	20.0%	25.7%	24.6%	23.3%	23.0%
Preventive conservation	62	37	54	40	41	34	268
Percentage	33.2%	21.8%	31.8%	23.4%	24.6%	25.6%	26.9%
Scientific analysis	76	102	82	87	85	68	500
Percentage	40.6%	60.0%	48.2%	50.9%	50.9%	51.1%	50.1%
Total	187	170	170	171	167	133	998

* An additional 256 scientific analysis services were conducted internally to support other CCI services.

Source: PIMS standardized reports, July, 2016.

Table 5.8: list of Canadian heritage institutions having received ten or more than ten completed projects, 2010-2011 to 2015-16

Canadian Heritage	77
National Gallery of Canada	61
Art Gallery of Ontario	35
Canadian Museum of History	24
Musée de la civilisation (Québec)	24
Prince of Wales Northern Heritage Centre	23
McCord Museum	22
Royal Ontario Museum	21
Public Works and Government Services Canada	19
Musée national des beaux-arts du Québec	18
Laboratoire et la Réserve d'archéologie du Québec	17
Montreal Museum of Fine Arts	17
Canadian Private Individuals	14
Library and Archives Canada	14
Musée d'art contemporain de Montréal	13
Musée Royal 22e Régiment	11
Centre de conservation du Québec	10

Source: PIMS standardized reports, July 2016

Table 5.12: administrative flexibility of Canada government SOAs (excluding CCI), 2017

Organization	In-house finance function	In-house human resources function	Other noted in-house functions
Communications Research Centre	Finance management	n/a	Procurement Contracting IT applications development
Canadian Intellectual Property Office	Finance, facilities and security directorate	Strategic human resources center	Information and knowledge Management directorate
Measurement Canada	Financial services	Human resources	Web publishing
Competition Bureau	Finance and administration directorate	Talent management and development directorate	n/a
Office of the Superintendent of Bankruptcy	Revenue and accounts receivable agent	HR management officer	n/a
Indian Oil and Gas Canada	Financial services	Human resources	Information technology procurement

Appendix E: expert services offered by CCI

• Conservation and treatment services
  • Archaeological Field Service
  • Archaeology Treatment at CCI – Excavation
  • Conservation Examination at CCI
  • Conservation Examination at client site
  • Conservation Treatment at CCI
  • Conservation Treatment at client site
• Preventive conservation
  • Collection Survey / Risk Assessment Survey
  • Emergency Response
  • Equipment Loan
  • Facilities Advisory Service
• Scientific analysis
  • Scientific Service at CCI - Object Analysis and Examination
  • Scientific Service at CCI - Product and Material Evaluation
  • Scientific Service at client site
  • Other Professional Service

Appendix F: comparative table

Criteria	Mandate	Activities
Canadian Conservation Institute	Through its expertise in conservation science, restoration and preventive conservation, CCI supports and promotes conservation of Canadian heritage collections. CCI works with heritage institutions and professionals to ensure that these collections are preserved and made accessible to Canadians now and in the future. CCI operates in three main areas of activity, which constitute its program mandate: (1) ongoing research and development in conservation, (2) conservation and consulting service, and (3) knowledge transfer in the field of conservation. Footnote 41	Research and development Expert services Knowledge dissemination
Library and Archives Canada	Library and Archives Canada (LAC) combines the collections, services and staff of two former institutions, the National Library and National Archives of Canada. As stipulated in the preamble to the Library and Archives of Canada Act, LAC’s mandate is to: preserve documentary heritage for present and future generations; be a source of enduring knowledge accessible to all, contributing to the cultural, social and economic development of Canada as a free and democratic society; facilitate in Canada cooperation among communities involved in the acquisition, preservation and dissemination of knowledge; serve as the continuing memory of the federal administration and its institutions. Footnote 42	Conservators also provide advice and support to many of the other fundamental activities of LAC in the form of research into conservation techniques and approaches, advice on methods of disaster preparedness and recovery, and technical evaluations of samples and products. Footnote 43 Conservation of Books, Textual, and Visual Material - Treatments performed on documents by professional conservators vary from minimal interventions, intended to make as many items available for consultation as possible, to more extensive treatments when an item will be used for exhibition, publication or other special use. Activities range from the physical examination of new acquisitions to detailed treatments on single items.... Preservation of Audiovisual Recordings - The audiovisual preservation laboratories at LAC are amongst the largest of their kind in Canada. Working with specialized equipment, conservators handle all types of film, video and sound documents including numerous obsolete formats. Documents are stabilized, repaired, cleaned and inspected, and, when required, new preservation copies are made... Digitization - Digital imaging technicians reproduce a wide variety of archival and library material - for individual clients, Web exhibitions, and for LAC's program of mass digitization in support of its access agenda. Preservation of Digital Documentary Heritage - To meet the challenge of preserving digital collections in a rapidly changing social and technological environment LAC is developing a suite of Trusted Digital Repository services for its digital collection: ingest, storage, management, and access to. Footnote 44 Conservators also provide advice and support to many of the other fundamental activities of LAC in the form of research into conservation techniques and approaches, advice on methods of disaster preparedness and recovery, and technical evaluations of samples and products. Footnote 45
Parks Canada	The mandate of the Agency, on behalf of the Canadian people, is to protect and display representative examples of Canada’s natural and cultural heritage and to promote public awareness, appreciation and enjoyment of them in ways that sustain their ecological and commemorative integrity for present and future generations. Parks Canada is also mandated to commemorate Canada’s history. Based on recommendations from the Historic Sites and Monuments Board of Canada, the Minister designates places, persons and events of national historic importance. Currently, the Agency administers 168 national historic sites. In addition, Parks Canada represents Canada on the United Nations World Heritage Committee and manages 11 of the 17 World Heritage sites in Canada. Footnote 46	The Heritage Conservation and Commemoration Directorate is responsible for: ... programs for motivating Canadians to conserve historic sites, including the Canadian Register of Historic Places, the Standards and Guidelines for Conservation of Historic Places in Canada and the National Historic Sites Cost-Sharing Program. Footnote 47 The Directorate is responsible for: historical research relating to PC historic places; archaeological digs and underwater searches at PC sites; collection, preservation and conservation of objects at the national level. As custodian of 31 million archaeological and historic objects, PC actively manages and maintains this collection. The Agency restores and repairs heritage assets and takes conservation measures in situ to minimize deterioration and damage to historic and archaeological relics. It restores and stabilizes historic structures and buildings. Certain artifacts are displayed and interpreted at PC sites so that Canadians can gain a better understanding of their country’s history and relate more closely to it. Management of 31 million archaeological and historic objects; support for the Historic Sites and Monuments Board of Canada in its role as advisor to the Minister on the designation of historic places, people and events; Canadian leadership in implementation of international conventions, programs and agreements, including the United Nations World Heritage Convention; designation and commemoration programs, including: the work of the Federal Heritage Buildings Review Office, the National Program for the Grave Sites of Canadian Prime Ministers, and the Canadian Heritage Rivers Program. Footnote 48
Centre de conservation du Québec	The Centre de conservation du Québec is the only provincial government agency with a heritage conservation mandate. It was established in 1979 with the mission of contributing to the preventive conservation and restoration of Quebec’s heritage, comprising such things as objects, works of art, furniture and public art. Specifically, this means: providing professional restoration services and expertise; educating museums and other institutions on the importance of preventive conservation; encouraging development of the knowledge, skills and amenities needed for improved heritage conservation. Footnote 49	The institution’s mission is to contribute to the preventive conservation and restoration of Quebec’s heritage, meaning objects, works of art, furniture, public art, etc. The Centre’s intent is to improve the accessibility and appreciation of Quebec’s cultural heritage. Specifically, the Centre’s mandate is to: provide professional restoration services and expertise; educate museums and other institutions on the importance of preventive conservation; encourage development of the knowledge, skills and amenities needed to improve heritage conservation. Footnote 50 Advice on preventive conservation – the museum community, custodians of collections, artists, maintenance staff and individual owners can all benefit from advice on preserving their collections. Footnote 51 Training – this may involve lectures, hands-on workshops, customized courses designed to inculcate the skills and motivation to perform preventive conservation. Footnote 52 The Centre de conservation du Québec helps train restorers by taking on trainees each year. Footnote 53 On-line publications: free archiving guides and capsules Footnote 54; publication of CCQ works to order Footnote 55 and free specialized textsFootnote 56. Préserv'Art: interactive database of products and equipment used in conserving and protecting objects, works of art and archival documents; examination and condition reporting services: development of procedures and working tools for documenting conservation measures taken on objects or works: care record for the object or work, register of actions taken and maintenance records; and diagnosis and recommendations for restoration. Footnote 57