Evaluation of the Contribution to the Biological and Chemical Defence Review Committee

June 2014

1258-215 (CRS)

Reviewed by CRS in accordance with the Access to Information Act (AIA). Information UNCLASSIFIED.

Acronyms and Abbreviations

Executive Summary

The Evaluation of the Contribution to the Biological and Chemical Defence Review Committee (BCDRC, or “the Committee”) was conducted by the Chief Review Services (CRS) to assess the relevance and performance of the BCDRC, as part of an assessment for funding renewal. The current term of the Contribution to the BCDRC is effective until March 31, 2015.

Program Description

The BCDRC is currently composed of three academics in the fields of physiology, pharmacology and anaesthesia; microbiology; and chemistry. The BCDRC Executive Officer is a civilian external to DND; the Program Manager is the Director National Defence Headquarters (NDHQ) Secretariat. Committee members are primarily involved in inspecting and visiting establishments that engage in activities involving BCD, with a view to gathering pertinent information and documenting and developing recommendations by means of a publicly-available annual report. Annual spending directly attributed to the BCDRC through Vote 10 expenditures averaged $113,870 over the period of fiscal year (FY) 2008/09 to FY 2012/13.

Relevance

The evaluation team found that the Contribution to the BCDRC remains relevant, as the BCDRC supports obligations under international conventions, and that DND/CAF BCD-related activities in Canada are transparent. It is also an important mechanism for risk mitigation. Furthermore, by providing assurance to the Canadian public (and the international community) that Canada is respecting provisions that BCD-related activities are strictly defensive in nature, the BCDRC was found to be well aligned with DND’s roles and responsibilities. The Evaluation also established that the Program is well aligned with the Government of Canada’s priority of transparency, accountability and responsiveness.

Performance

Evidence indicates that the anticipated outcomes of the BCDRC are being realized. The Program produces information that is publicly accessible, and knowledge is effectively transferred from Committee members to key stakeholders. Through visits to and inspections of DND BCD-related research and development facilities (among other activities), the BCDRC confirms that BDC-related activities are defensive in nature and pose minimal risk to the public and the environment. The production of an annual report on a publicly-accessible website is a significant contribution by the BCDRC to the continued transparency of Canada’s BCD-related activities. Other important results attributable to the Program are that it contributes to improvements to Canada’s BCD-related programs and to fostering public confidence that DND/CAF BCD-related activities are for defensive purposes only, and are conducted in a safe and professional manner. The evaluation team established that the BCDRC successfully contributes to the provision of independent assurance of compliance with policies, treaties and obligations.

Compared to the cost of the contribution to the BCDRC, the benefits to key program stakeholders, the Canadian public and the international community were found to be significant and economical. Program spending is being well managed and spent to achieve stated outcomes through targeted activities.

Key Findings and Recommendations

Finding 1. Relevance—Continued Need.

There is a continued need for independent expert review of Canada’s BCD-related programs.

Finding 2. Relevance—Alignment with Federal Roles and Responsibilities, and Government Priorities.

The BCDRC is aligned with both DND’s roles and responsibilities, and the government of Canada’s priority of transparency, accountability and responsiveness.

Finding 3. Immediate Outcome—Public Availability/Dissemination of Information.

The BCDRC products and information are publicly accessible.

It is recommended to document and track over time consultations held with community members and interested citizens’ organizations in annual reports.

Finding 4. Immediate Outcome—Knowledge Transfer.

The Program transfers knowledge effectively through multiple means. Challenges to the knowledge transfer process are mostly beyond the BCDRC’s control.

It is recommended that primary stakeholders are notified when the BCDRC Annual Report is published.

Finding 5. Immediate Outcome—Confirmation that BCD-related Programs are Defensive in Nature and Pose Minimal Risk.

The BCDRC activities contribute to the confirmation that DND/CAF BDC-related activities are defensive in nature and pose minimal risk to the public and the environment.

Finding 6. Intermediate Outcome—Continued Transparency.

The BCDRC contributes to the continued transparency of Canada’s BCD-related activities. The Committee’s greatest contribution is the provision of information through the BCDRC Annual Report on a publicly-accessible website.

Finding 7. Intermediate Outcome—Improvements to BCD-related Programs.

The BCDRC contributes to improvements to Canada’s BCD-related programs relative to the scope of the Committee’s activities.

It is recommended to continue advising DND/CAF stakeholders on the intended readership of the BCDRC Annual Reports to ensure that the language is clear and easily understandable to the general public.

Finding 8. Intermediate Outcome—Fostering Public Confidence.

The Program contributes to fostering public confidence that DND/CAF’s BCD-related activities are for defensive purposes only and are conducted in a safe and professional manner.

Finding 9. Ultimate Outcome—Independent Assurance of Compliance with Policies, Treaties and Obligations.

The BCDRC contributes to the provision of independent assurance of compliance with policies, treaties and obligations.

Finding 10. Efficiency of Output Delivery.

Program expenditures have remained relatively stable over the years. Little variation in spending by service area year-over-year indicates costs are being well managed.

Finding 11. Adequacy of Progress Toward Achieving Expected Outcomes with Resources Expended.

Some program efficiencies could be gained but these would likely not improve program outcomes.

In conclusion, it is recommended that the Contribution to the BCDRC be renewed in 2015.

Note: Refer to Annex A—Management Action Plan—for a complete list of recommendations and management responses.

1.0 Introduction

1.1 Profile of the Biological and Chemical Defence Review Committee

1.1.1 Background

This report presents the findings and recommendations of the evaluation team about the Contribution to the BCDRC. The current Contribution Agreement between the BCDRC and DND expires on March 31, 2015. In accordance with the Policy on Transfer Payments¹ of Treasury Board (TB), an evaluation is required to support the renewal of the Program’s terms and conditions.

CRS conducted a previous evaluation of the Contribution to the BCDRC in 2009, prior to the Contribution Agreement’s renewal in 2010. Where relevant, the current evaluation utilizes those results as a baseline to assess continuous improvement.

1.1.2 Program Description

In the late 1980s, responding to protests, concerns raised,² and suspicion over the development of offensive weaponry, then-Minister of Defence Perrin Beatty commissioned William H. Barton (former chief disarmament negotiator), to study the activities of DND in the area of biological and chemical warfare. The Barton Report (1988) made numerous recommendations, one of which was the establishment of an advisory committee of scientists to annually visit DND facilities and review its biological and chemical programs. The succeeding Minister, Bill McKnight, established the BCDRC (“the Committee”) in May 1990.

The BCDRC was initially established under the administration of the Defence Science Advisory Board, with operations supported under Vote 1 funds. In 1997, the BCDRC was separated from this Board, and to strengthen the arm’s-length relationship between the BCDRC and DND/CAF, a contribution agreement using Vote 10 funds to support the BCDRC’s operations was established. The BCDRC is self-administering, reporting to the NDHQ Corporate Secretary for administrative matters only.

The mandate of the BCDRC is to review DND/CAF’s BCD programs involving research and development, and training, and assess whether BCD-related programs are defensive in nature; compliant with international treaties and obligations; and conducted in a professional manner, and pose a minimal threat to the public or the environment.

The BCDRC Committee is currently composed of three academics in the fields of physiology, pharmacology and anesthesia; microbiology; and chemistry. The BCDRC Executive Officer is a civilian external to DND; the Program Manager is the Director of the NDHQ Secretariat.

1.1.3 Program Objectives

The overarching objective of the BCDRC is to provide tangible evidence and foster public confidence that DND/CAF’s BDC programs are for defensive purposes only, as well as to provide independent expert review to improve DND/CAF BCD-related programs.

In the Contribution Agreement (2010) between the BCDRC and the Minister of National Defence, the BCDRC committed to:

• receive briefings from DND/CAF and other stakeholders on BCD-related program issues;
• inspect some associated (mostly DND) government research and development facilities, such as Defence Research and Development Canada (DRDC) Suffield;
• preselect and visit other CAF training establishments, operational formations and units where BCD activity occurs; and
• publish annual reports in the public domain with key observations, findings and recommendations; and review responses from DND/CAF.

The specific activities, outputs and outcomes of the BCDRC are illustrated in the Logic Model at Annex C.

The BCDRC is primarily involved in inspecting and visiting establishments where BCD activity occurs, gathering pertinent information, and documenting and developing recommendations in the form of a publicly-available annual report. These activities support the dissemination of information to the Canadian public, support the transfer of knowledge to DND/CAF stakeholders, and confirm that BCD-related programs are defensive in nature and pose minimal risk to the public and the environment. In turn, these outcomes are expected to contribute to the continued transparency of Canada’s BCD-related activities, improvements to BCD-related programs, and the fostering of public confidence. Ultimately, these outcomes are expected to contribute to the provision of assurance to the Canadian public and the international community that the Government of Canada’s policy of maintaining only a defensive capability in this field is fully respected at all times, and that any BCD-related research, development or training activities undertaken are:

• defensive in nature;
• compliant with international treaties and obligations as legislated by Canadian law; and
• conducted in a professional manner with minimal threat to public safety or the environment.

1.1.4 Program Stakeholders

As detailed in the Table below, the key stakeholder and the BCDRC’s target population is the Canadian public, and, to a lesser extent, the international community with respect to demonstrating Canada’s compliance with its arms control treaty obligations. Other beneficiaries may receive direct or indirect independent expert opinion for improving national BCD-related programs.

Table 1. List of Key Program Stakeholders

1.2 Evaluation Scope

CRS undertook the evaluation activities between September 2013 and March 2014. The Evaluation was conducted in accordance with the DND/CAF Five-Year Evaluation Plan 2013/14 to 2017/18, and examined the relevance and performance of the BCDRC for the period 2009 to 2014.

It should be noted that the evaluation study was limited to examining only BCDRC program-related activities specified within the Contribution Program. It did not examine all BCDRC-related activities supported by DND/CAF. The Vote 10 payments made under the BCDRC Contribution Program, in most cases, contribute specifically to the administrative component of the activities within the Program. Costs related to the employment of DND/CAF personnel in support of BCDRC activities are not covered by the BCDRC Vote 10 funds, and are not within the scope of this evaluation.

1.2.1 Coverage and Responsibilities

The BCDRC falls under the 2009 Program Alignment Architecture (PAA) 1.1.1 Research, Technology and Analysis. Approximately 0.03 percent of the total $1.7 billion allocated to this PAA element over the last five fiscal years (2008/09 to 2012/13) was expended on the Contribution to the BCDRC.³ The total amount payable to the BCDRC, as stated in the current Contribution Agreement, is $610,800 over five years (2010 to 2015).

The delegated Program Manager is the Director NDHQ Secretariat, who, through the NDHQ Corporate Secretary, is responsible to the Deputy Minister and the Chief of the Defence Staff for departmental administration of the BCDRC Contribution Program. To ensure the Committee’s autonomy, the Program Manager has no mandate for program delivery, which is solely the Committee’s responsibility. The Program Manager has authority to approve Contribution payments for eligible expenditures approved by the TB Secretariat and stated in the Funding Agreement. The Program Manager is also responsible for monitoring ongoing compliance with the Terms and Conditions set out in the Funding Agreement. The BCDRC Chairperson⁴ signs the Funding Agreement with the Program Manager. The recipient is responsible to conduct BCDRC activities as described in the Agreement, maintain the required financial records, and publish an annual report (that is publicly available).

1.2.2 Resources

Annual spending directly attributed to the Contribution to the BCDRC averaged $113,870 over the period FY 2008/09 to FY 2012/13 (Vote 10 funds).⁵

1.2.3 Issues and Questions

In accordance with the TB Directive on the Evaluation Function (2009), the Evaluation addressed five core issues related to relevance and performance. The methodology used to gather evidence in support of the evaluation questions is provided at Annex B.

The issues and questions addressed in the Evaluation are as follows (refer to Annex D for the complete evaluation matrix, which also includes specific indicators and methodologies for each question).

Relevance

• Continued Need. To what extent does the BCDRC address a demonstrable need and is responsive to the needs of Canadians?
• Alignment with Federal Roles and Responsibilities. How do the roles and responsibilities of the federal government align with the BCDRC?
• Alignment with Government Priorities. How does the BCDRC align with federal government priorities and departmental strategic outcomes?

Performance (Effectiveness)—Immediate Outcomes

• Does the BCDRC produce and disseminate publicly available information on BCD-related program activities?
• Does the BCDRC effectively transfer knowledge to DND/CAF stakeholders with respect to the policy compliance, professional conduct, and safety of BCD-related programs?
• Does the BCDRC adequately confirm that BCD-related programs are defensive in nature and pose minimal risk to the public and the environment?

Performance (Effectiveness)—Intermediate Outcomes

• To what extent does the BCDRC contribute to the continued transparency of Canada's BCD-related activities?
• Does the BCDRC contribute to improving BCD-related programs? How and to what degree?
• Does the BCDRC foster public confidence that BCD-related activities are for defensive purposes only and are conducted in a safe and professional manner?

Performance (Effectiveness)—Ultimate Outcomes

• To what extent does the BCDRC contribute to assuring the Canadian public and the international community that:
  • a) the Government of Canada’s policy of maintaining only a defensive capability in this field is fully respected at all times;
  • b) that any BCD-related research, development or training activities undertaken are:
    • defensive in nature;
    • compliant with international treaties and obligations;
    • conducted in a professional manner; and
    • pose a minimal threat to public safety or the environment.

Performance (Efficiency and Economy)

• Is the DND/CAF delivering programs/services to [end users] in an efficient manner? What is being spent on by the BCDRC?
• Is the progress made toward expected outcomes adequate for the resources expended?

Other Issues

• Have there been any unintended outcomes?
• What progress has been made in relation to findings of the 2009 evaluation report?

2.0 Findings and Recommendations

The following sections examine the extent to which the BCDRC continues to address a demonstrable need and is responsive to the needs of Canadians; how well the Program aligns with federal roles and responsibilities, as well as government priorities and DND strategic outcomes; and the extent to which the Program has achieved expected outcomes. Finally, an assessment of the BCDRC’s performance in terms of efficiency and economy is provided.

To make this determination, the following indicators were assessed:

• analysis of empirical data (program activities and results), including secondary sources of information for public opinion research (i.e., Ministerial Inquires, media analyses, and Access to Information requests); and
• assessment of qualitative evidence provided by documents reviewed for the Evaluation, as well as the opinions of consulted stakeholders.

2.1 Continued Need for the BCDRC

The finding is based on documents reviewed, stakeholders consulted, and public opinion research.

Key Finding 1: There is a continued need for independent expert review of Canada’s BCD-related programs. The BCDRC’s greatest contributions are supporting obligations under international conventions, supporting the transparency of DND/CAF BCD-related activities in Canada, and serving as a mechanism for risk mitigation.

The BCDRC allows Canada to confirm the ongoing implementation of its obligations as a signatory party to two conventions: the Chemical Weapons Convention (CWC) and the Biological and Toxin Weapons Convention (BTWC).⁶ The obligations under these conventions support the ongoing need for independent expert review to confirm Canada’s compliance via-à-vis the prohibition of BCD weapons:

• The CWC contains measures related to increased “transparency of national programs related to protective purposes” and to “assigning the highest priority to ensuring the safety of people and to protecting the environment.”
• The BTWC aims to prohibit any biological weapons “in quantities that have no justification for prophylactic, protective or other peaceful purposes.”

Most stakeholders consulted during the Evaluation felt there is a continued need for an independent expert review of DND/CAF BCD-related activities; only a few disagreed. Specifically, the necessity of reassuring the public and maintaining an arms-length oversight was mentioned, as well as the continued existence of biological and chemical-related threats (as confirmed by recent world events). The Evaluation was advised that the independent expert review is useful to DND/CAF as it is a mechanism for risk mitigation. Without such oversight, concern by the public of BCD-related activities could (re)emerge.

Among the stakeholders who were uncertain as to the need for the BCDRC, the specific need for public reporting was questioned, and potential duplication with other BCD-related oversight activities was pointed out (such as the verification system of the CWC⁷). Changes in Canadian public viewpoints since the 1980s were also stressed: a change in “security acceptance,” whereby the general public is more accepting of classified information; and a greater trust that regulatory tools are in place to ensure Canada’s compliance with treaties and conventions. A few stakeholders pointed out that, based on the current low awareness of the BCDRC among the Canadian population, its need may be questionable.

Based on a public opinion research conducted for the Evaluation, there appears to be relatively little public concern with respect to BCD-related activities during the last decade or so. This was the finding from a review of Canadian media articles, questions posed in Parliament, Access to Information requests to DND, and DND Ministerial Inquiries.

The Evaluation established, through a review of Canadian media articles, that there has been no mention of the BCDRC in the Canadian media since 2003.⁸ Although no questions have been raised in Parliament on the BCDRC specifically (in the period 2006–2014), nor about “biological defence” or “chemical warfare,” some interest was demonstrated in biological and/or chemical weapons, as detailed in the following Table.

Table 2. Summary of Parliamentary Questions Posed and Keyword Searches

However, some evidence of public interest was found on the subject of biological and/or chemical defence-related activities in reviewing Access to Information requests to DND. From 2009 to 2013, five such requests were submitted on subjects including either “biological” or “chemical” as keywords.¹⁰ In reviewing DND Ministerial Inquiries for the 2008 to 2013 period, the Evaluation found 61 Ministerial Inquiries that were submitted on the topic of BCD or BCD-related activities,¹¹ approximately half of which had a “negative” tone (that is, inquiries were of a suspicious or discordant nature¹²).

Although there currently appears to be limited public interest in BCD-related activities in Canada generally, evidence suggests that there remains at least some concern; this attests to a continued need for the BCDRC. As suggested by stakeholders consulted for the Evaluation, the fundamental need for the BCDRC stems from its function as a risk-mitigation mechanism, without which DND/CAF could be vulnerable to public outcry as it was in the late 1980s.

2.2 Alignment with Federal Roles and Responsibilities, and Government Priorities

The finding in this section is based on documentation reviewed and interviews with stakeholders consulted for the evaluation.

Key Finding 2: By providing assurance to the Canadian public (and the international community) that Canada is respecting provisions that BCD-related activities are strictly defensive in nature, the BCDRC is aligned with both DND’s roles and responsibilities, and the government of Canada’s priority of transparency, accountability and responsiveness.

The Canadian federal government states its obligation to protect the security of its citizens,¹³ and the BCDRC verifies that BCD-related activities are conducted professionally and with minimal risk to the public, which corresponds with the protection of Canadian citizens. Most stakeholders consulted agreed that it is appropriate for DND to be funding the BCDRC, though the importance of the arms-length relationship was underlined. The evaluation assesses that the current program design–a committee composed of independent experts, funded by DND–is appropriate.

It is a federal government priority to be transparent and to safeguard the public trust in government,¹⁴ which the BCDRC contributes to directly. Furthermore, DND is mandated to meet the country’s defence needs and support the government’s foreign policy and the contribution to the BCDRC can be considered as meeting part of that objective.

The scientific knowledge and innovation generated from the activities undertaken by the BCDRC informs decisions on Defence capability acquisitions, readiness preparation and the conduct of operations in response to Government priorities. Activities under this program draw on internal capability and make extensive use of partnerships with Canadian industry and academia as well as international organizations. This activity is aligned with the Government of Canada outcome, "A transparent, accountable and responsive federal government.”¹⁵

The Canada First Defence Strategy states that the CAF is expected to “meet the country’s defence needs, enhance the safety and security of Canadians and support the Government’s foreign policy”; the BCDRC, as funded by DND, is a contribution to realizing those objectives as it supports Canada’s standpoint as a signatory party to conventions banning non-defensive BCD activities.

2.3 Achievement of Expected Outcomes (Effectiveness)

The following sections provide findings as to whether the Contributions to the BCDRC have achieved intended results over the course of the evaluation period. To determine the overall effectiveness of the Contributions to the BCDRC, a logic model of the program was developed. The logic model grouped the program activities by common outputs, and then linked outputs to intended outcomes (See Annex C).

The effectiveness of the Contributions to the BCDRC was assessed by applying appropriate performance measures and/or key performance indicators against each expected outcome. Data for the performance measures was obtained from reports, documents, and financial information provided by the program stakeholders. Based upon analysis of this information, the overall effectiveness of the Contributions to the BCDRC was determined. Accordingly, an assessment was made of the following outcome areas:

Immediate outcomes

• Public Availability/Dissemination of Information
• Knowledge Transfer
• Confirmation that BCD-related Programs are Defensive in Nature and Pose Minimal Risk

Intermediate outcomes

• Continued Transparency
• Improvements to BCD-related Programs
• Fostering Public Confidence

Ultimate outcomes

• Independent Assurance of Compliance with Policies, Treaties and Obligations

2.3.1 Public Availability/Dissemination of Information

The finding in this section is based on documentation reviewed and interviews with stakeholders consulted by the evaluation team.

Key Finding 3: The BCDRC products and information are publicly accessible.

The BCDRC contributes to the public availability of information on BCD-related activities in Canada through a variety of means, including the BCDRC Annual Report which is published on a publicly-accessible website.¹⁶

Though the BCDRC website is clearly publicly accessible, assessing awareness among the Canadian (and international) population is somewhat challenging. An analytics report of the BCDRC website¹⁷ is used by BCDRC staff to measure awareness through access to the website. This report calculates that, in a 17-month period (April 2012 – September 2013), 539 individuals visited the BCDRC website (average duration of 3 ½ minutes). Results also indicate that during that period, forty or so individuals from outside Canada accessed the website (i.e., USA, China, UK, Germany, etc.). Without a target or a benchmark, however, it is difficult to situate this result.

As a contribution to the public accessibility of the program and to disseminate pertinent information, the BCDRC maintains a Wikipedia entry.¹⁸ Committee members also hold occasional workshops and contacts with the community (i.e., communities surrounding DRDC Suffield). Though the evaluation found that such consultations contribute to program outcomes (i.e., public availability/dissemination of information, knowledge transfer, transparency, public confidence), it is of interest to note that they are not defined as part of the core activities of the program. Furthermore, these types of consultations were reported in BCDRC Annual Reports in earlier years, but consultation details have not been documented since 2000.

As part of their defined program activities, Committee Members conduct scheduled briefings, visits and related activities, to facilities where BCD-related activities take place and document these in the BCDRC Annual Report, such as that detailed in the 2012 Annual Report:

Table 3. Summary of Briefings, Visits and Related Activities, and Corresponding Locations

The program is operating within its mandate and above its mandate, as per consulting with community representatives and is achieving its short-term outcome since the information generated by the BCDRC is publicly accessible.

2.3.2 Knowledge Transfer

The findings in this section are based on documentation reviewed and interviews with stakeholders.

Key Finding 4: The program transfers knowledge effectively through multiple means. Challenges to the knowledge transfer process are mostly beyond the BCDRC’s control.

Many stakeholders consulted for the evaluation believe the BCDRC effectively transfers knowledge, mainly through the BCDRC Annual Report which is published on the BCDRC website, but also through consultations with stakeholders (visits, briefings, and other professional exchanges).

Stakeholders consulted for the evaluation recommended that it would be useful to be notified of the publication of the BCDRC Annual Report (via a distribution list), rather than leave it up to them to estimate when the report might be available online. Readership would increase, it was predicted, if this task were undertaken. Currently, several direct stakeholders admitted they do not read the BCDRC Annual Report but asserted they would if they knew when it was being published.

The evaluation conducted an analysis of the recommendations provided in the BCDRC Annual Reports. Since the start of the program in 1990, the BCDRC annual reports have provided 93 recommendations in total (one report yearly, except 2010,¹⁹ up to 2012). These reports clearly provided the status of the recommendations: 24 were implemented or otherwise closed, and five were not.²⁰ Reports dropped another 34 recommendations stating that they could be assumed to be complied with to the satisfaction of the Committee. The 2011 Annual Report dropped all impending recommendations (16 in total), stating that they had been satisfactorily resolved. Following that, 12 recommendations remained open, and were introduced in the 2011 and 2012 reports. Based on these results, the evaluation concludes that DND stakeholders have implemented most of the BCDRC recommendations.

The following table presents a summary of the recommendations that have not been implemented by DND. Among these, two were not resolved due to organizational changes: the “National Medical Decontamination Platoon” was not established and the Public Safety and Emergency Preparedness Canada entity moved from DND to Public Safety Canada. The reasons provided by DND for not resolving the other recommendations are vaguer: fiscal constraints, and unsuccessful international discussions.

Table 4. Summary of BCDRC Recommendations not Implemented and DND’s Response

2.3.3 Confirmation that BCD-related Programs are Defensive in Nature and Pose Minimal Risk

Key Finding 5: The BCDRC activities contribute to the confirmation that DND/CAF BDC-related activities are defensive in nature and pose minimal risk to the public and the environment.

Most stakeholders consulted for the evaluation believe that the BCDRC adequately confirms that DND/CAF BCD-related activities are defensive in nature and pose minimal risk to the public and the environment. Some potential gaps were identified: the BCDRC cannot be entirely thorough in their assessments; and, they only see part of what happens at any given time, particularly since visits to facilities are scheduled. Furthermore, some facility accessibility issues were mentioned in interviews such as Committee members not always having the required security clearance or the proper vaccinations. However, the evaluation notes that these (and related) “risks” have been identified in the BCDRC Risk Assessment Framework (2009), and mitigation strategies have been put in place to address them.

Consulted stakeholders reported some safety measures have been adopted following BCDRC recommendations, such as the improved management of stocks and inventories, changes to relic toxin access and waste management, evacuation plans and exercises, and changes in practices at the Royal Military College of Canada.

Through their activities, the BCDRC directly contributes to confirming that DND/CAF BDC-related activities are defensive in nature and pose minimal risk to the public and the environment.

2.3.4 Continued Transparency

Key Finding 6: The BCDRC contributes to the continued transparency of Canada’s BCD-related activities. The Committee’s greatest contribution is the provision of information through the BCDRC Annual Report on a publicly-accessible website.

Most consulted stakeholders perceived that the BCDRC contributes to the continued transparency of Canada's BCD-related activities. When asked to provide examples of the BCDRC's contribution, the BCDRC's Annual Reports were mentioned, as well as the BCDRC website and the composition of the Committee itself. Among the few who did not share this perception, it was proposed that an outreach/educational initiative would further transparency of BCD-related activities, such as in the United States where there is public knowledge of how to respond to chemical, biological, radiological-nuclear attacks (emergency response).

BCDRC staff and Committee members were consulted on the amount of information not disseminated to Committee members by program stakeholders or limitations with respect to information that could be included in Annual Reports due to security issues. It was unanimously stated that only a small portion of the information related to visits and/or inspections is omitted due to security reasons, and that a certain level of detail is omitted from BCDRC Annual Reports due to context and privacy concerns.

With respect to the continued transparency of Canada's BCD-related activities, the BCDRC’s greatest contribution is ensuring that their annual reports are easily accessible on the internet. These reports add a facet of transparency to Canada’s BCD-related activities.

2.3.5 Improvements to BCD-related Programs

Key Finding 7: The BCDRC contributes to improvements to Canada’s BCD-related programs relative to the scope of the Committee’s activities.

Many of the stakeholders consulted for the evaluation believe that the BCDRC contributes to improving Canadian BCD-related programs as a whole. They indicated that the Committee improved BCD-related-practices, mitigated risks and focused the attention of higher level DND stakeholders on BCD-related issues, leading to action when necessary.

There was a perception among stakeholders consulted for the evaluation that the implementation of recommendations can be a lengthy process, particularly when other government departments are involved. To verify this perception, the evaluation conducted an analysis of the recommendations in BCDRC Annual Reports and DND/CAF’s responses. Recommendations with proposed timelines for implementation and/or other Departments involved were examined to establish the length of time it took to either table, close or implement the recommendation. On the whole, no significant delays were noted by the evaluation, and recommendations involving other Departments did not cause particular delays in the implementation of recommendations.

Consulted stakeholders mentioned that there are some challenges with respect to DND/CAF stakeholders responding to BCDRC recommendations. It was pointed out that DND/CAF should take into account that the readership of the Annual Report includes the general public and therefore responses should not use acronyms or vague responses such as “our staff is responding”.

The evaluation was advised by BCDRC staff that this matter was brought to the attention of DND/CAF prior to the development of their response to recommendations in the 2013 Annual Report and, consequently, the DND/CAF’s responses were much improved.

2.3.6 Fostering Public Confidence

Key Finding 8: The program contributes to fostering public confidence that DND/CAF BCD-related activities are for defensive purposes only and are conducted in a safe and professional manner.

As noted in Section 2.1 (Relevance), there has been relatively low public concern with respect to BCD-related activities in Canada in recent years. One could make an assumption that low public concern is an indication that the public is confident that BCD-related activities are for defensive purposes only and are conducted in a safe and professional manner. However, without a baseline for the degree of public confidence, nor results tracked over time, the current level can only be speculated.

Despite not knowing the actual level of public confidence, the evaluation found that through their activities (mandated and not mandated), the Committee does in fact foster public confidence: the BCDRC presents tangible evidence reinforcing that Canada’s BCD-related activities are for defensive purposes only; it provides independent expert review of these programs which contributes to the improved safety of BCD-related activities; and, the Committee responds to community members and interested citizens’ organizations requests for consultations on BCD-related matters.

2.3.7 Independent Assurance of Compliance with Policies, Treaties and Obligations

Key Finding 9: The BCDRC contributes to the provision of independent assurance of compliance with policies, treaties and obligations, though the extent is difficult to establish.

Although the public opinion research conducted for this evaluation did not provide clear evidence of the need for independent expert review of BCD-related activities in Canada (see Section 2.1 Relevance), there is a perception that the program contributes to the ultimate outcome:

[Provision of ] independent assurance to the Canadian public and the international community that the Government of Canada’s policy of maintaining only a defensive capability in this field is fully respected at all times and, that any BCD-related research, development or training activities undertaken are defensive in nature; compliant with international treaties and obligations as legislated by Canadian law; and, conducted in a professional manner with minimal threat to public safety or the environment ²¹ .

Most of the stakeholders consulted for the evaluation believe the BCDRC does in fact contribute to this outcome. Many noted the absence of concern by the public. It was also pointed out that Canada has been held up as an example of a useful review mechanism during the course of arms control discussions. The article “Critical Guidance: A Code of Conduct for Biodefense Scientists” published by the Arms Control Association²² proposes that it is essential to couple codes of conduct with independent mechanisms that could provide oversight to assure the public that biodefense programs are purely defensive. The article refers specifically to the BCDRC in this context.

Thus, evidence found during the course of the evaluation established that the BCDRC does contribute to the provision of assurance, but the extent remains difficult to establish.

2.4 Demonstration of Efficiency and Economy

The findings in the following sections are based on an assessment of the efficiency of the BCDRC’s delivery of outputs, what the program is spending, and whether progress made toward expected outcomes is adequate for the resources expended.

2.4.1 Efficiency of Output Delivery

The finding in this section is based on program information reviewed and stakeholders consulted for the evaluation, and financial information.

Key Finding 10: Program expenditures have remained relatively stable over the past 5 years. There has been little variation in spending by service area year-over-year indicating costs are being well managed. In some instances, expenditures by service area have varied from the budget projections, however given the overall size of the budget, and that overall spending is within 2 percent of the program budget over the 5-year period, these variances are not significant.

The following graph presents BCDRC Vote 10 expenditures and the amounts payable to the program via the 5-year Contribution Agreement. Spending directly attributed to the BCDRC through Vote 10 expenditures averaged $113,870 over the period FY 2008/09 to FY 2012/13,²³ and remained relatively stable over the period examined. On average, the program spent within 2 percent of its overall projected budget (excluding FY 2010/11 and FY 2013/14).

Figure 1. BCDRC Expenditures and Amounts Payable This figure illustrates the distribution of Vote 10 BCDRC Expenditures and amounts payable as per the Contribution Agreement for FY 2008/09 to FY 2013/14. The data is summarized in Table 5.

Table 5. BCDRC Expenditures and Amounts Payable

As indicated in the following table, program expenditures by year have varied from the budget for specific service areas (as indicated by a variance value²⁴). For FY 2011/12 and FY 2012/13,²⁵ more was spent on administration and management than planned, but by FY 2013/14 the budget was more or less on target. Actual spending on visits, briefings, inspections and other verification activities cost less than budgeted for FY 2011/12 to FY 2013/14. Likewise, less was spent on the Annual Report than planned for both FY 2012/13 and FY 2013/14. However, only around one-third of what was budgeted to produce the Annual Reports for FYs 2012/13 and 2013/14 was budgeted for in FY 2011/12 ($10,000 vs. $30,000). This resulted in a dramatic underestimation of the cost of producing the Report in FY 2011/12 (variance of +250 percent). The evaluation team observes that some efficiency could be gained by revising budget expectations for the main program service areas.

Table 6. Actual and Budgeted Program Expenditures by Service Area

Table 7 presents the percentage of costs expended on each service area over the total expenditures for the year. The program spends, on average, 55 percent of its expenditures on producing outputs related to visits, briefings, inspections, and other verification activities, and 17 percent on producing the Annual Report.²⁶ On average, 28 percent is spent on administration and management, including the cost of planning and executing visits, maintaining the BCDRC website, and associated per diems and travel expenses for the BCDRC Executive Officer specifically.

Table 7. Percentage Costs Expended on Service Areas of Total Expenditures

With respect to administration and management costs, such expenses varied between 24 percent and 34 percent between FY 2008/09 and FY 2012/13.²⁷ Given how expenses covered under this service area are defined (i.e., attributable per diems and travel expenses), it is not feasible to benchmark these overhead costs to those incurred in other similar programs or organizations. However, since there has been relatively little variation over the years, it can be concluded that overhead costs are being relatively well managed.

2.4.2 Adequacy of Progress Toward Achieving Expected Outcomes with Resources Expended

The finding in this section is based on stakeholder interviews, document review and comparative analysis.

Key Finding 11: Some program efficiencies could be gained but these would likely not improve program outcomes.

The evaluation team examined programs similar to the BCDRC to determine whether there were comparable indications of efficiencies to be gained or best practices that could be used by the BCDRC. The evaluation discovered that the Public Health Agency of Canada National Microbiology Laboratory (in Winnipeg) involves an oversight mechanism by Health Canada’s Population and Public Health Branch,²⁸ and also meets with a community oversight group four times a year in an effort to proactively report incidents. The evaluation also found that the province of Quebec has established a National Public Health Institute which oversees main public health laboratories and centres of expertise.²⁹ The BCDRC’s mandate and operations differ to such a degree from these programs, however, that comparisons are not feasible.

The Australian Safeguards and Non-Proliferation Office³⁰ (ASNO) and the Australian Biosecurity Intelligence Network (ABIN)/Connecting Spaces program³¹ were also reviewed for the evaluation. On the whole, the ASNO was found to differ to a great degree from the BCDRC. Unlike the arm’s length nature of the BCDRC, the ASNO is staffed through the Australian Department of Foreign Affairs and Trade. The ASNO also has an extensive Australian on-site consultation and outreach program aimed at raising awareness of affected parties of CWC obligations, whereas the BCDRC does not have an outreach component (other than occasional consultations with “concerned citizens”, as discussed in Section 2.4.1). Furthermore, the ASNO Annual Report is very different from the BCDRC product. The ASNO Annual Report deals mostly with disarmament issues and world events. It is an extensive report, and not focused solely on Australia. It includes a section on the performance of ASNO in implementing the CWC, as well as Advice to Government (similar to recommendations to DND by the BCDRC).

The ABIN, situated within the Australian Department of Industry, is a government-funded research infrastructure project. The aim of this project spans human, animal, wildlife, plant and aquatic animal health and provides expertise, ease of communication and linked data for those involved in research, surveillance, preparedness and emergency responses. The initial focus of ABIN was biosecurity, although its mandate has changed to include the capacity to serve broader secure communication and collaboration needs, including those of the wider “Safeguarding Australia” community. Its financial model is very different from BCDRC, as it uses a fee-for-service operational model.

In exploring whether efficiencies could be gained by reducing program expenditures, stakeholders were consulted. Most shared the opinion that the outcomes of the Program are appropriate and adequate, given the resources expended. Approximately $100,000 a year was considered a small amount to pay relative to the value of the BCDRC Annual Reports, and the fact that the Committee serves as a mechanism for risk mitigation.

In researching alternatives on program delivery, the evaluation team also explored whether alternative methods could achieve the same objectives at the same or lower cost. It was suggested by evaluation stakeholders that some planning and travel costs could be reduced by conducting more consultations by teleconference. BCDRC staff pointed out, however, that it is important to have on-site visits, otherwise the thoroughness of the visits and inspections might be compromised.

There are no specific instructions in program guidelines or directives with regard to the number of sites for visits/inspections to be conducted annually; this decision is left to the discretion of the BCDRC. A few consulted stakeholders suggested that the same program results might be achieved with a fewer number of visits, and even with the production of the BCDRC Report on a bi-annual basis. The need to conduct on-site visits/inspections every year was questioned, and it was proposed that a risk-based approach might be preferable (selecting sites where/when risks or concerns arise). It is challenging to measure the specific impact that changes in these program activities would have on program outcomes, but the likelihood that an impact would occur in at least two of the immediate outcomes is high. Specifically, there would likely be an impact on public availability/dissemination of information, and on knowledge transfer. Furthermore, an impact on these immediate outcomes would theoretically lead to an impact on intermediate and ultimate outcomes as well.

Given the size/scope of this Contribution program, and its current level of success in the achievement of outcomes, the evaluation team concludes that the benefits to be gained by implementing some efficiencies would not outweigh the risk of compromising BCDRC’s contributions to program outcomes.

2.5 Conclusion

The evaluation team determined that the BCDRC remains relevant, and continues to serve an ongoing need. The program was found to be well aligned with federal roles and responsibilities, as well as with government priorities. Through conducting annual inspections and visiting BCD-related research and development facilities (among other activities), the BCDRC successfully confirms that BCD-related activities are defensive in nature and pose minimal risk to the public and the environment. Information generated from the BCDRC is publicly available, and the Program successfully transfers knowledge to key stakeholders. Furthermore, the Program has contributed to the continued transparency of Canada’s BCD-related activities and to fostering public confidence, and has led to improvements to the BCD-related program in Canada.

The BCDRC has been administered efficiently, and indicates a good level of economy for the DND/CAF in consideration of program costs against the benefits to the DND/CAF, as well as the Canadian public and (to a lesser degree) the international community. In conclusion, the BCDRC is effectively achieving all anticipated outcomes, and provides good value for money.

Annex A—Management Action Plan

Management Action Plan

The Terms and Conditions of the BCDRC are silent on the issue of consultations with community members and interested citizens and the Committee does not proactively pursue consultation with them. However, the Committee will consider meeting with any citizen or group that approaches them and any consultations that take place will be documented in the annual report.

OPI: NDHQ Corporate Secretary

Target Date: None specified/ongoing

Management Action Plan

This recommendation is fully supported. The Committee with post their annual report on their website (www.bcdrc.ca). The Departmental Program Manager will distribute hard copies internally as per the established distribution list and will issue a broadcast e-mail notification of the publication of the report with the address of the website included.

OPI: NDHQ Corporate Secretary

Target Date: Within 30 days of the publication of the annual report

Management Action Plan

The Committee will include a reminder to this effect in its letter to the Minister covering the submission of the advance copy of the Annual Report, wherein responses are requested to the recommendations.

OPI: NDHQ Corporate Secretary

Target Date: When the annual report is submitted to the Ministry of National Defence for consideration

Management Action Plan

The Departmental Program Manager will work with the Committee to negotiate appropriate revisions to the terms and conditions and renew the BCDRC Contribution Program in accordance with the TB Policy on Transfer Payments.

OPI: NDHQ Corporate Secretary

Target Date: End-March 2016

Annex B—Evaluation Methodology and Limitations

1. Methodology

After the initial scoping and the design and development of performance measurement tools—the program logic model and the evaluation matrix—in consultation with program stakeholders, the methodology was established to provide multiple lines of evidence, both qualitative and quantitative, in support of the findings on relevance and performance. This methodology was selected to ensure consistency in the collection and analysis of data addressing performance indicators, as detailed in the evaluation matrix based on the Logic Model of the BCDRC program.

1.1 Overview of Data Collection Methods

The Evaluation of the BCDRC included the use of multiple lines of evidence and complementary research methods to help ensure the reliability of information and data collected. The following data collection methods were used to gather qualitative and quantitative data:

• document review and comparative analysis;
• stakeholder interviews;
• secondary data research (public opinion); and
• financial data review.

Where appropriate, the data obtained from lines of evidence was analyzed using a customized template organized according to evaluation questions and corresponding key performance indicators. Each of these methods is described in more detail in the following sections.

1.1.1 Document Review and Comparative Analysis

Program documentation was reviewed as part of the planning and scoping stage to identify stakeholders and build an accurate logic model and evaluation matrix. Government of Canada and DND documents (e.g., Speech from the Throne, the Canada First Defence Strategy, DND’s Report on Plans and Priorities, the DND Departmental Performance Report, etc.), and international treaties, namely the CWC and the BTWC, were subsequently reviewed.

Available documentation on similar programs was researched, with the intent to assess best practices and possible alternatives (i.e., efficiencies gained). The Evaluation examined the ASNO and ABIN/Connecting Spaces.

1.1.2 Stakeholder Interviews

Initial consultations were conducted with program staff members as part of the scoping of the Evaluation. Stakeholders were identified and appropriate interview guides were designed, one for the program staff and Committee members, and one for all other internal and external program stakeholders. In total, 21 interviews (with 23 interviewees) were conducted. The Program Director, the BCDRC Executive Officer and all four members of the Committee were interviewed first, followed by internal and external stakeholders. The internal DND/CAF stakeholder organizations were:

• Assistant Deputy Minister (ADM) Policy
• BCD-related training OPIs for the Canadian Army, Royal Canadian Air Force and the Royal Canadian Navy
• Chief of Military Personnel
• Canadian Special Operations Forces Command
• Assistant Deputy Minister (Science and Technology)
• Chief of Defence Intelligence
• Director General Health Services
• ADM Finance and Corporate Services
• ADM Public Affairs
• DRDC
• Director NDHQ Secretariat
• Director Chemical, Biological, Radiological and Nuclear Defence

The external stakeholders were from DFATD Canada.

Interview questions were aligned with the evaluation questions and performance indicators identified in the matrix. A semi-structured method was selected, with interviews lasting about an hour, in person or by phone, in each interviewee’s language of preference (French or English). Two interviewers took notes whenever possible, although some interviews were conducted by a single interviewer. Transcripts were reviewed by both interviewers when applicable.

1.1.3 Secondary Data Research (Public Opinion)

Public opinion research was conducted to assess the intermediate outcome “Public confidence that BCD-related activities are for defensive purposes only and are conducted in a safe and professional manner.” Access to Information requests, Ministerial Inquiries and media coverage related to the BCDRC or BCD activities in Canada were examined.

Measures for public confidence ideally would have included obtaining perspectives from the general public. Due to the sensitivity of the issue and the scope of the Evaluation, proxy sources of information were used to assess public confidence (media sources, ministerial requests, Access to Information requests, etc.).

1.1.4 Administrative and Financial Data Review

An analysis was conducted to assess resource utilization (per output) of the BCDRC. Consultations were held with the BCDRC Executive Officer for information on the amount of spending on program expenditures for the evaluation period (five years). Projected versus actual expenditures were also reviewed, to assess effectiveness in financial management of the program. The data obtained was compiled in an Excel spreadsheet and analyzed by year.

1.1.5 Limitations

The evaluation methodology was designed to provide multiple lines of evidence in support of findings. The data and information were collected to respond to the evaluation questions and issues. As with all evaluations, there are limitations and considerations that should be noted. Thus, constraints and limitations of the Evaluation were considered.

In every evaluation, the determination of causality requires evidence as to whether the program itself is causing the changes that are observed, or whether events or processes outside of the program may be the real cause of the observed outcome or the prevention of an anticipated outcome. In the case of the BCDRC, there are obviously other events and processes impacting outcomes—such as improvements to Canada’s BCD-related programs, and fostering public confidence that BCD-related activities are for defensive purposes only. To mitigate this issue, the evaluation team was alert to evidence of clear indicators of effect pertaining to specifically to the BCDRC’s contributions.

Another important limitation is that the measure for public confidence ideally would have included obtaining perspectives from the general public. A mitigation strategy used to address this limitation is the use of proxy sources of information to assess public confidence (media sources, Ministerial Inquiries, Access to Information requests, etc.).

Annex C—Logic Model

Figure C-1. Logic Model for the BCDRC. This logic model shows how program inputs, activities and outputs link to immediate, intermediate and ultimate (governmental) outcomes.

Annex D—Evaluation Matrix

Table D-1. Evaluation Matrix—Relevance

Table D-2. Evaluation Matrix—Performance: Achievement of Expected Outcomes (Effectiveness)

Table D-3. Evaluation Matrix—Relevance and Performance (Efficiency and Economy)

___________________________________________________________________________________________________________________________

Footnote 1 http://www.tbs-sct.gc.ca.

Footnote 2 Mainly by citizens’ groups such as The Calgary Disarmament Coalition, Science for Peace, and Canadian Physicians for the Prevention of Nuclear War.

Footnote 3 Totaling $478,000 of Vote 10 funds (excluding Vote 1).

Footnote 4 A Committee Member is selected by the Committee to serve as Chair. New members are appointed by the Chair on the basis of nominations from professional societies and associations. The Chair also arranges for an administrative staff member to function as the Committee’s Executive Officer.

Footnote 5 This calculation excludes FY 2010/11—when a delay in renewing the Contribution Agreement had an impact on the delivery of the Annual Report (on average 17 percent of the program’s expenditures)—and FY 2013/14, as only partial financial information was available at the time of the Evaluation.

Footnote 6 The complete names of these conventions are: Convention on the Prohibition of the Development, Production and Stockpiling of Bacteriological (Biological) and Toxin Weapons and on their Destruction; and Convention on the Prohibition of the Development, Production, Stockpiling and Use of Chemical Weapons and on their Destruction.

Footnote 7 Under the CWC, each state party is obliged to accept international verification of its compliance with the provisions of the Convention. The verification system of the CWC is based upon each state party providing declarations that are verified by the Organization for the Prohibition of Chemical Weapons through data monitoring and on-site routine inspections. There is also a provision for challenge inspections. (www.international.gc.ca/arms-armes/non_nuclear-non_nucleaire/chemical-chimique.aspx).

Footnote 8 Four articles were published in Canadian newspapers in the period 2000-2003, all mentioning the BCDRC specifically.

Footnote 9 A chlorinated hydrocarbon commonly used as an industrial solvent; a clear non-flammable liquid with a sweet smell (Wikipedia).

Footnote 10 These were: 1) Toxic trial scenarios and chemical/biological response field exercises at DRDC Suffield chemical training site; 2) Documentation related to training for chemical biological warfare in the 50s, 60s, and 70s. Types and details of chemical(s), information on their manufacturing; DND standing orders, policy and/or protocol pertaining to training, and Workplace Hazardous Material Information System information; 3) Records relating to aerial spraying in British Columbia airspace by DND; 4) Email correspondence (between DRDC Suffield and DFATD); and 5) Records of asbestos removal/modification/repair documents and air-borne breathing quality/pollution reports for the Canadian Forces Fleet School Halifax/Fire Fighting School Damage Control Division.

Footnote 11 Only four of these specified the representative organizations submitting the Inquiries, including a federal department, a private company, an academic department, and a church. Topics of inquiry included: BCDRC, biological, chemical, defence, warfare, weapon, phosphorus, gas, bacteria, toxic, virus, terror, and/or organic.

Footnote 12 As determined by DND analysts within the Director Strategic Corporate Services, Corporate Secretary.

Footnote 13 Canada’s National Security Policy (2004).

Footnote 14 “Our Government will also ensure that citizens, the private sector and other partners have improved access to the workings of government through open data, open information and open dialogue.” (Speech from the Throne, 2011, page 15); “A transparent, accountable and responsive federal government:Program activities aim to safeguard public trust in government and strengthen the interaction between the government and Canadian citizens. This is done through the delivery of services that are responsive to citizens’ needs, and through public accountability, compliance and recourse measures.” TBS (www.tbs-sct.gc.ca).

Footnote 15 DND, Departmental Performance Report 2011/12.

Footnote 16 www.bcdrc.ca. Note that the website was moved from DND/CAF’s Defence Wide Area Network to a public platform in 2012.

Footnote 17 Google Analytics: Provided by BCDRC staff.

Footnote 18 http://en.wikipedia.org/wiki/BCDRC.

Footnote 19 A delay in renewing the Contribution Agreement impacted the delivery of the Annual Report in 2010.

Footnote 20 However, only four recommendations are provided in Table 4, as one recommendation appears to be a duplication.

Footnote 21 BCDRC Logic Model in Annex C.

Footnote 22 www.armscontrol.org.

Footnote 23 Calculation excluding FY 2010/11, as a delay in renewing the Contribution Agreement impacted the delivery of the Annual Report, on average 17 percent of the program’s expenditures. Calculation also excludes 2013/14 as financial information for that period is only until February 24, 2014.

Footnote 24 Budgeted expenses were not available for years prior to FY 2011/12.

Footnote 25 BCDRC Staff have not been able to locate a statement of projected expenditures for FY 2008/09, FY 2009/10 and FY 2010/11. BCDRC staff explained that while the Contribution Agreement in effect during those years makes reference to "receipt of the Committee's annual plan" after which the first installment of the Contribution was to be paid, there was no mention of a requirement for spending estimates. The current Contribution Agreement also stipulates submission of an “annual visit and inspection plan" which BCDRC staff have interpreted as including a spending estimate for each activity in the plan. Accordingly, they produced an Activity and Estimated Expenditure Plan for FY 2011/12, FY 2012/13 and FY 2013/14.

Footnote 26 Includes attributable per diems and travel expenses for each activity category.

Footnote 27 Excluding FY 2010/11 and FY 2013/14.

Footnote 28 Which includes a component for Centres for Infectious Disease Prevention and Control, Chronic Disease Prevention and Control, Emergency Preparedness and Response, Surveillance Coordination, and Healthy Human Development.

Footnote 29 Its mandate covers prevention, community development and health promotion, healthy living, workplace health, and chronic disease as well as infectious diseases. It includes the Quebec Toxicology Centre, the Screening Expertise Centre, and the Poison Control Centre. www.inspq.qc.ca.

Footnote 30 http://www.dfat.gov.au/asno/about_us.html.

Footnote 31 http://www.innovation.gov.au/science/ResearchInfrastructure/FundedProjects/Pages/ABIN.aspx#5.