ARCHIVED - Summative Evaluation of the Blood Safety Contribution Program - Final Report

IV. Summary of key findings

The BSCP’s lines of accountability and accountability measures are reasonably well documented and clear, based on the review of the program documents.

However, there are diverse opinions among stakeholders regarding the clarity of the program’s lines of accountability and accountability measures. At least half of the working group members and funding recipients feel that the accountability measures for the program are not totally clear.

One major issue for PHAC staff and provincial/territorial representatives is the delay in renewing contribution agreements. This delay has a number of negative impacts, including, delayed data receipt from provinces/territories, financial hardship on the provinces/territories, and disengagement among stakeholders.

Stakeholder participation appears to have been relatively high (with a few exceptions), as evidenced by attendance at committee meetings.

While stakeholders noted that the BSCP had engaged them sufficiently, there was still room for further stakeholder engagement efforts. Some other ways that BSCP stakeholders could be engaged include: sharing further information on all BSCP activities; continuing to provide education documents; building linkages with other organizations; renewing contribution agreements more efficiently; and streamlining both the submission form and reporting process.

TTISS
Transfusion-related adverse event surveillance systems have been established in all but one province and one territory. Data for the remaining territory is captured through one of the provinces. The remaining province has been engaged, but a system has not yet been established in that province.

At a national level, and in all but one of the participating provinces/territories, TTISS is capturing 70% or more of all transfusion activity (70% is the current benchmark for TTISS reporting, according to program staff). TTISS also appears to capture a larger proportion of adverse events than pre-existing reporting systems, such as hospitals reporting to Canadian Blood Services and the Marketed Health Products Directorate.

The TTISS does not appear to be overly complex. Most hospitals, as well as most provinces/territories, seem to be willing to participate.

A number of supports, as well as checks and balances, have been implemented to improve the accuracy of the data. Sites are generally using the national standards and tools, and also appear to investigate and confirm the adverse events before reporting them to TTISS. Despite these efforts, there continue to be some issues with the completeness and accuracy of the data. Currently, provincial/territorial representatives follow up with front line staff to obtain more accurate or complete information.

The data are being analysed and reported back to stakeholders. Key stakeholders on working groups also review the data to identify issues and trends. There are concerns that it takes too long to issue reports (at the national level) to make it possible to identify issues and trends in a timely manner.

TESS TESS has made substantial progress towards becoming a fully operational national surveillance system with sentinel characteristics; this outcome may be fully achieved within the next year or so.

As of December, 2005, 3 small^{Footnote 4}, 5 medium, and 3 large sites in four provinces were participating in TESS, representing 10% of all transfusions that occur in Canada. There was no consensus from the interviews about whether these are enough sites to permit relevant generalizations to other hospitals in Canada. For example, some stakeholders feel that more small sites are needed.

The TESS system does not appear to be overly complex. However, the error coding system could possibly be streamlined further. Most of the involved hospitals, and all of the involved provinces/territories, seem to be willing to participate without signs of resistance.

There are a number of supports, as well as checks and balances, that have been implemented to improve the accuracy and consistency of the data. Resource limitations prevent on-site training and limit the capacity for data quality assessment; however, this evaluation was unable to determine if these constraints had a negative impact on data quality.

At this point, the TESS data are being analysed at the national level, but no surveillance products have been published.

CTOSS
CTOSS is still in the planning stage, and has not yet been implemented, so the outcome of ‘a fully operational national cells, tissues and organs transplant surveillance system’ has not yet been achieved.

Public Health Linkages
The provinces and territories have begun to establish linkages with public health that enable identification of delayed adverse events. However, they have met with varying degrees of success, and not all have developed informal or formal relationships with public health. This outcome has not yet been fully achieved.

Blood Sample Archive
A blood sample archive for high risk populations has been established, and continues to be added to. Although several projects related to high risk populations have received funding, only one project, the Blood Borne Pathogens Surveillance Project (BBPSP), has resulted in an ongoing blood sample archive for high risk populations. BBPSP has archived blood from patients with bleeding disorders for the last 8 years, and from patients with other disorders for the last 3 years. A total of 2,000 patients have been included, from 23 sites in nine provinces.

Risk Identification for High Risk Populations
The BBPSP and TTISS have laid the groundwork for risk identification for high risk groups, but the identification of these risks is not yet being carried out in a systematic manner.