ARCHIVED - Summative Evaluation of the Blood Safety Contribution Program - Final Report


Executive Summary


Each year in Canada, over 1.2 million units of red cells, plasma and platelets are distributed; 2000 organs are transplanted and over 90,000 tissue allografts are distributed for transplantation. Thus, effective surveillance of these procedures is an important aspect of public health safety.

The Blood Safety Contribution Program (BSCP) was introduced to coordinate and implement surveillance of transfusion of blood and blood products, and transplantation of cells, tissues, and organs. The BSCP has an annual budget of $4 million. It provides national tools and standards for surveillance, provides funds for provinces and territories to establish and maintain surveillance activities, and coordinates the analysis and interpretation of the surveillance data. The BSCP includes five components:

  1. Transfusion Transmitted Injuries Surveillance System (TTISS): surveillance systems in each province/territory for transfusion-related adverse events;
  2. Transfusion Errors Surveillance System (TESS): a national sentinel system for monitoring transfusion-related errors;
  3. Cells, Tissues and Organs Surveillance System (CTOSS): a surveillance system (still under development) for transplant-related adverse events and errors;
  4. Data linkages and communications with public health to capture adverse events outside of the hospital reporting system; and
  5. High risk populations and emerging pathogens: collection of blood samples from high risk populations to enable identification of new/emerging pathogens.

Purpose, Scope and Methods of the Evaluation

The Public Health Agency of Canada (PHAC) undertook this summative evaluation of the BSCP, as per its federal obligations, to assess its relative impact on health surveillance activities, as well as its impact on the safety of the Canadian blood supply. This evaluation was national in scope, including surveillance activities in all 13 provinces and territories. It covers the period from October 1998 to April 2008, and was carried out between April and November 2008. Methods for the evaluation included a literature review, review of program documents, surveys of key program stakeholders, key informant interviews, and an expert panel.

Evaluation Questions and Key Findings

The following table provides an overview of specific answers to each of the evaluation questions:

Table 1: Overview of Answers to Specific Evaluation Questions
Evaluation Question Brief AnswerTable 1 - Footnote 1*
*There is a high level of confidence in the completeness of the answers to most of the evaluation questions. However, there is less confidence in the completeness of the answers to question 12, 13 and 14 due to pragmatic constraints of the evaluation project itself (i.e., scope, budget, availability of data, etc.).
1. Is there a continued need for the BSCP? Yes, BSCP’s purpose is still as relevant as ever.
2. Are the BSCP target audiences and activities appropriate in light of societal/environmental requirements? Yes, target audiences and activities are appropriate.
3. Does the program continue to be consistent with departmental and government-wide priorities? Yes, but it could benefit from‘re-alignment’ with more pressing priorities that are currently receiving greater attention, such as example patient safety.
4. What are the best practices and lessons learned from the development and implementation of the five components that make up the BSCP?   Challenges/Lessons Learned:
  • Lack of recognition for BSCP accomplishments,
  • Data quality (for TTISS and TESS),
  • Under-utilization of data,
  • Departmental bureaucracy,
  • Lack of consistent strategic leadership,
  • Under-resourcing of CTOSS, and
  • IT demands.
Opportunities/Best Practices:
  • Collaborative relationships,
  • Canada’s role as a world leader in blood safety, and
  • Potential for recognition and support within Canada.
5. Are the accountability measures for the three surveillance systems and lines of accountability clear to stakeholders, including funding recipients? Lines of accountability and accountability measures are reasonably well documented, however, they are not entirely clear to all stakeholders.
6. To what extent has the BSCP been successful in engaging its stakeholders For the most part, BSCP has been successful in engaging its stakeholders. Delay in agreement renewal and funding negatively impacts engagement.
7. To what extent have the short-term outcomes of the BSCP been achieved?
  1. A fully operational adverse event surveillance system in each province/territory
  2. A fully operational national error surveillance system
  3. A fully operational national cells, tissues and organ transplant surveillance system
  4. Linkages with Public Health Information Networks (PHINs) established
  5. Blood sample archive for high risk populations established
  6. Risk identification for high risk populations established

TTISS is almost fully operational.

TESS is almost fully operational as a sentinel system.

CTOSS is not yet operational.

Some progress has been made to establish relevant linkages with PHINs in several P/Ts.

A blood sample archive for high risk populations has been established.

Risk identification for high risk populations is occurring to some extent.
8. What remains to be done before each component of the BSCP is fully operational? TTISS
  • Fully engage all P/Ts, including capture of at least 70% of transfusion activity
  • Address ongoing data quality issues
  • More timely reporting
  • National level reports need to be produced
  • Determine number & type of sites needed to be representative
  • Engage more sites if necessary
  • Streamlining of error coding system may be needed
  • Planning, resourcing, piloting, refining, etc.
Linkages with PHINs
  • Relationships need to be established consistently in all P/Ts
Risk identification for High Risk Populations
  • The Blood Borne Pathogens Surveillance Project has only done baseline testing – agreements need to be reached to enable further testing and identification of risks
Do more analyses of TTISS data related to high risk groups
9. / 10. What progress has been made towards the intermediate and long-term outcomes of the BSCP?

a) Enhanced capacity to identify and assess risks (IO) and develop mitigating strategies (LTO)

b) Enhanced capacity to identify and assess risks (IO) and develop mitigating strategies
(LTO) for high risk populations

Risks have been identified, assessed, and mitigated for transfusion-related adverse events/errors. Mitigating strategies include the development of guidelines for investigation of suspected bacterial contamination and guidelines for identifying TACO and TRALI. Risks have not yet been identified, assessed, and mitigated for transplant-related adverse events/errors.

Risks for high risk populations have reportedly been identified, assessed, and mitigated for transfusion-related adverse events, but no specific examples were provided. Risks have not yet been identified, assessed, and mitigated for transfusion-related errors or transplant-related adverse events/errors.
11. Have there been any unintended (positive or negative) outcomes of the BSCP activities? If so, what are they, and what actions were taken as a result? Positive
  • International recognition of Canada as a leader in hemovigilance
  • Development of communication network between PHAC, HC, provinces/territories, and blood manufacturers
  • Hospital staff better at recognizing and reporting adverse reactions à improved transfusion safety
  • No negative unintended outcomes were identified.
12. How have the contributions flowing to each of the funding recipients impacted the achievement of BSCP outcomes and contributed to national surveillance systems? The BSCP likely contributed to the development of surveillance capacity in the P/Ts. Most stakeholders thought it would be more difficult to identify and assess risks without the BSCP data.
13. Is BSCP as designed the most appropriate and efficient means of achieving its objectives? The BSCP appears to provide good value for money. Delays related to agreement renewal and funding negatively impact program efficiency.
14. What resources are required to sustain, strengthen or enhance blood safety surveillance? It is not clear precisely what resources will be required. However, the annual overall costs of maintaining a surveillance system may actually be the same or higher than the initial annual start-up costs.

Overarching Conclusions

There continue to be real risks to patient safety as a result of transfusion and transplantation. It is important for the federal government to take a lead role in biovigilance at both the national and international levels.

The BSCP already has some impressive accomplishments. In particular, the TTISS and TESS systems are almost fully operational, and have contributed to improved patient safety due to increased recognition of adverse events in hospitals. In light of the low cost of the program and the accomplishments to date, stakeholders feel that the BSCP provides very good value for money.

The BSCP has a few outstanding tasks to make TTISS and TESS fully operational, and CTOSS still needs to be developed and implemented. In addition, there are opportunities, both within Canada and internationally, for the BSCP to maximize its contribution to patient safety.

Key Observations

The following is a list of key observations flowing from the evaluation findings, presented in order of priority.

  1. Strategic positioning of the BSCP within government priorities should be considered.  The program may enjoy greater support if the Canadian government truly understood what was being accomplished. It will be important that BSCP accomplishments be communicated and promoted more heavily and consistently within the federal government.
  2. The BSCP requires dynamic upper level leadership with a strong background in blood safety and/or a keen interest in and capacity to develop blood safety expertise who can set a bold future course for the program, raise the program’s profile, and leverage opportunities within Canada and internationally;
  3. Canada should take advantage of the worldwide respect for the BSCP to raise the country’s profile in the international blood safety community. Fully leveraging this opportunity may require targeted resources, conscious strategic planning and positioning, as well as strong responsive leadership with experience and profile in the hemovigilance field. PHAC will need to identify a strong, knowledgeable spokesperson for the BSCP whose responsibility it is to represent Canada effectively and consistently in international forums and provide advice and support to other countries as requested.
  4. A review of the contribution project approval process within PHAC should be undertaken to address the inefficiencies that have been experienced by staff and recipients.  This should include a review of procedural requirements within PHAC to mitigate unnecessary bureaucracy, in line with the recommendations resulting from the Blue Ribbon Panel.  In addition, PHAC should review whether contribution funding is the most appropriate method where long-term funding is required to maintain the surveillance systems once they have been established.


The following is a list of recommendations flowing from the evaluation findings, presented in order of priority.

  1. Greater effort could be made to emphasize and promote the link between ‘blood safety’ and ‘patient safety’, to underscore and refresh the consistency of the BSCP with broader government and department priorities that are currently receiving significant attention.
  2. A Strategic Human Resource Plan should be developed and implemented by PHAC over the next twelve months, taking into account the current and future needs of the program to support achievement of its full potential. This may include focussing on the identification and recruitment of:
    • In-house expertise in blood safety and/or transfusion medicine (particularly among epidemiologists);
    • In-house IT expertise;
    • In-house statisticians (particularly those capable of sophisticated modelling) who will be able to make the most of the existing data, and improve the timeliness of analysis and reporting; and
    • General support staff to reduce delays in agreement processing, funding, reporting, etc. It is acknowledged that implementation of this recommendation may have resource implications; however, specific cost implications cannot be estimated at this point. Further investigation and consideration will be required.
  3. Continue establishing and improving upon existing BSCP components:
    • For TTISS: ensure adequate participation of the provinces and territories, and work towards more timely reporting. Continue to improve data quality. Conduct analyses to enable risk identification for high risk populations.
    • For TESS: ensure adequate national representation among participating hospitals (particularly small hospitals), and ensure timely reporting. Also, consider further streamlining of the error coding system to promote simplicity and enhance data quality.
    • For CTOSS: this system still needs to be developed, piloted, refined, and implemented across the country. The feasibility of developing and implementing CTOSS, even if done incrementally, within the current allocated budget should be critically reviewed. Next steps (e.g., reallocating BSCP budget, request for additional funding) should be considered as appropriate. 
    • For public health linkages: these need to be fostered more consistently in the provinces and territories, as well as at the national level.
    • For the blood sample archive: agreements need to be reached about confidentiality and testing, so that testing can be carried out and results reported.

Page details

Date modified: