ARCHIVED - Summative Evaluation of the Blood Safety Contribution Program - Final Report
Each year in Canada, over 1.2 million units of red cells, plasma and platelets are distributed; 2000 organs are transplanted and over 90,000 tissue allografts are distributed for transplantation. Thus, effective surveillance of these procedures is an important aspect of public health safety.
The Blood Safety Contribution Program (BSCP) was introduced to coordinate and implement surveillance of transfusion of blood and blood products, and transplantation of cells, tissues, and organs. The BSCP has an annual budget of $4 million. It provides national tools and standards for surveillance, provides funds for provinces and territories to establish and maintain surveillance activities, and coordinates the analysis and interpretation of the surveillance data. The BSCP includes five components:
- Transfusion Transmitted Injuries Surveillance System (TTISS): surveillance systems in each province/territory for transfusion-related adverse events;
- Transfusion Errors Surveillance System (TESS): a national sentinel system for monitoring transfusion-related errors;
- Cells, Tissues and Organs Surveillance System (CTOSS): a surveillance system (still under development) for transplant-related adverse events and errors;
- Data linkages and communications with public health to capture adverse events outside of the hospital reporting system; and
- High risk populations and emerging pathogens: collection of blood samples from high risk populations to enable identification of new/emerging pathogens.
Purpose, Scope and Methods of the Evaluation
The Public Health Agency of Canada (PHAC) undertook this summative evaluation of the BSCP, as per its federal obligations, to assess its relative impact on health surveillance activities, as well as its impact on the safety of the Canadian blood supply. This evaluation was national in scope, including surveillance activities in all 13 provinces and territories. It covers the period from October 1998 to April 2008, and was carried out between April and November 2008. Methods for the evaluation included a literature review, review of program documents, surveys of key program stakeholders, key informant interviews, and an expert panel.
Evaluation Questions and Key Findings
The following table provides an overview of specific answers to each of the evaluation questions:
|Evaluation Question||Brief AnswerTable 1 - Footnote 1*|
|1. Is there a continued need for the BSCP?||Yes, BSCP’s purpose is still as relevant as ever.|
|2. Are the BSCP target audiences and activities appropriate in light of societal/environmental requirements?||Yes, target audiences and activities are appropriate.|
|3. Does the program continue to be consistent with departmental and government-wide priorities?||Yes, but it could benefit from‘re-alignment’ with more pressing priorities that are currently receiving greater attention, such as example patient safety.|
|4. What are the best practices and lessons learned from the development and implementation of the five components that make up the BSCP?||Challenges/Lessons Learned:
|5. Are the accountability measures for the three surveillance systems and lines of accountability clear to stakeholders, including funding recipients?||Lines of accountability and accountability measures are reasonably well documented, however, they are not entirely clear to all stakeholders.|
|6. To what extent has the BSCP been successful in engaging its stakeholders||For the most part, BSCP has been successful in engaging its stakeholders. Delay in agreement renewal and funding negatively impacts engagement.|
|7. To what extent have the short-term outcomes of the BSCP been achieved?
TTISS is almost fully operational.
TESS is almost fully operational as a sentinel system.
CTOSS is not yet operational.
Some progress has been made to establish relevant linkages with PHINs in several P/Ts.
A blood sample archive for high risk populations has been established.
Risk identification for high risk populations is occurring to some extent.
|8. What remains to be done before each component of the BSCP is fully operational?||TTISS
|9. / 10. What progress has been made towards the intermediate and long-term outcomes of the BSCP?
a) Enhanced capacity to identify and assess risks (IO) and develop mitigating strategies (LTO)
b) Enhanced capacity to identify and assess risks (IO) and develop mitigating strategies (LTO) for high risk populations
Risks have been identified, assessed, and mitigated for transfusion-related adverse events/errors. Mitigating strategies include the development of guidelines for investigation of suspected bacterial contamination and guidelines for identifying TACO and TRALI. Risks have not yet been identified, assessed, and mitigated for transplant-related adverse events/errors.
Risks for high risk populations have reportedly been identified, assessed, and mitigated for transfusion-related adverse events, but no specific examples were provided. Risks have not yet been identified, assessed, and mitigated for transfusion-related errors or transplant-related adverse events/errors.
|11. Have there been any unintended (positive or negative) outcomes of the BSCP activities? If so, what are they, and what actions were taken as a result?||Positive
|12. How have the contributions flowing to each of the funding recipients impacted the achievement of BSCP outcomes and contributed to national surveillance systems?||The BSCP likely contributed to the development of surveillance capacity in the P/Ts. Most stakeholders thought it would be more difficult to identify and assess risks without the BSCP data.|
|13. Is BSCP as designed the most appropriate and efficient means of achieving its objectives?||The BSCP appears to provide good value for money. Delays related to agreement renewal and funding negatively impact program efficiency.|
|14. What resources are required to sustain, strengthen or enhance blood safety surveillance?||It is not clear precisely what resources will be required. However, the annual overall costs of maintaining a surveillance system may actually be the same or higher than the initial annual start-up costs.|
There continue to be real risks to patient safety as a result of transfusion and transplantation. It is important for the federal government to take a lead role in biovigilance at both the national and international levels.
The BSCP already has some impressive accomplishments. In particular, the TTISS and TESS systems are almost fully operational, and have contributed to improved patient safety due to increased recognition of adverse events in hospitals. In light of the low cost of the program and the accomplishments to date, stakeholders feel that the BSCP provides very good value for money.
The BSCP has a few outstanding tasks to make TTISS and TESS fully operational, and CTOSS still needs to be developed and implemented. In addition, there are opportunities, both within Canada and internationally, for the BSCP to maximize its contribution to patient safety.
The following is a list of key observations flowing from the evaluation findings, presented in order of priority.
- Strategic positioning of the BSCP within government priorities should be considered. The program may enjoy greater support if the Canadian government truly understood what was being accomplished. It will be important that BSCP accomplishments be communicated and promoted more heavily and consistently within the federal government.
- The BSCP requires dynamic upper level leadership with a strong background in blood safety and/or a keen interest in and capacity to develop blood safety expertise who can set a bold future course for the program, raise the program’s profile, and leverage opportunities within Canada and internationally;
- Canada should take advantage of the worldwide respect for the BSCP to raise the country’s profile in the international blood safety community. Fully leveraging this opportunity may require targeted resources, conscious strategic planning and positioning, as well as strong responsive leadership with experience and profile in the hemovigilance field. PHAC will need to identify a strong, knowledgeable spokesperson for the BSCP whose responsibility it is to represent Canada effectively and consistently in international forums and provide advice and support to other countries as requested.
- A review of the contribution project approval process within PHAC should be undertaken to address the inefficiencies that have been experienced by staff and recipients. This should include a review of procedural requirements within PHAC to mitigate unnecessary bureaucracy, in line with the recommendations resulting from the Blue Ribbon Panel. In addition, PHAC should review whether contribution funding is the most appropriate method where long-term funding is required to maintain the surveillance systems once they have been established.
The following is a list of recommendations flowing from the evaluation findings, presented in order of priority.
- Greater effort could be made to emphasize and promote the link between ‘blood safety’ and ‘patient safety’, to underscore and refresh the consistency of the BSCP with broader government and department priorities that are currently receiving significant attention.
- A Strategic Human Resource Plan should be developed and implemented by PHAC over the next twelve months, taking into account the current and future needs of the program to support achievement of its full potential. This may include focussing on the identification and recruitment of:
- In-house expertise in blood safety and/or transfusion medicine (particularly among epidemiologists);
- In-house IT expertise;
- In-house statisticians (particularly those capable of sophisticated modelling) who will be able to make the most of the existing data, and improve the timeliness of analysis and reporting; and
- General support staff to reduce delays in agreement processing, funding, reporting, etc. It is acknowledged that implementation of this recommendation may have resource implications; however, specific cost implications cannot be estimated at this point. Further investigation and consideration will be required.
- Continue establishing and improving upon existing BSCP components:
- For TTISS: ensure adequate participation of the provinces and territories, and work towards more timely reporting. Continue to improve data quality. Conduct analyses to enable risk identification for high risk populations.
- For TESS: ensure adequate national representation among participating hospitals (particularly small hospitals), and ensure timely reporting. Also, consider further streamlining of the error coding system to promote simplicity and enhance data quality.
- For CTOSS: this system still needs to be developed, piloted, refined, and implemented across the country. The feasibility of developing and implementing CTOSS, even if done incrementally, within the current allocated budget should be critically reviewed. Next steps (e.g., reallocating BSCP budget, request for additional funding) should be considered as appropriate.
- For public health linkages: these need to be fostered more consistently in the provinces and territories, as well as at the national level.
- For the blood sample archive: agreements need to be reached about confidentiality and testing, so that testing can be carried out and results reported.
- Date modified: