ARCHIVED - Summative Evaluation of the Blood Safety Contribution Program - Final Report

Appendix A: Detailed Key Findings

This section presents the key findings for each of the evaluation questions.

Efficiency of Program Design

Evaluation question 13: Is BSCP as designed the most appropriate and efficient means of achieving its objectives?

With any government program, it is important to determine if taxpayers’ money is being well spent. This section presents findings that reveal the extent to which the program design and delivery provide good value for money, in light of the program goals.

Program Spending

In 2007-2008, the budgeted cost of the BSCP was $3.92 million. As shown in Table 4, program operations and salaries together accounted for just under 40% of the budgeted program costs, while contributions funding accounted for just over 60%.

The budgeted costs of the program differ somewhat from the actual spending. The annual actual contribution funding has ranged from $1.5 million in 2004/2005 to just over $2 million in 2007/2008 (Log of BSCP contribution amounts, 2008). Table 5 displays the annual allocation, the amounts available to BSSHCAID after the PHAC annual budget reduction exercise^{Footnote 11}, the amounts BSSHCAID planned to distribute through contribution agreements, and the actual expenditures for all contribution agreements.

 

In general:

• The amounts available to BSSHCAID had been close to 100% of the annual allocation until 2007/08, when the amount decreased to 93%.
• BSSHCAID’s planned contributions varied by year, between 81% and 97% of the annual allocation. No particular trend was evident over time.
• The amount of funds actually distributed ranged between 63% and 87% of the annual allocation. Again, there was no particular trend over time.

PHAC staff indicated that they do make efforts to use the full funding allocated to the program. Program documents (e.g., Log of BSCP contribution amounts, 2008) suggest that several factors intervene:

• The contribution agreements are sometimes delayed. Expenses incurred by the recipients before signing the agreements are not covered. The contribution agreements also have fixed end dates, so even if activities (and expenses) are delayed until the agreement is signed, the recipients may not be able to use the full amount of funding in the remaining time.
• Recipients are sometimes unable to spend the full allocation, even when the contribution agreement has not been delayed. Recipients only receive funding for actual expenditures.
• Contribution agreements sometimes fall through altogether.
• Some of the funds are diverted for the PHAC budget reduction exercise.

Use of Program Funds

Since 2004-05, 16 different recipients have received funding for a total of 37 different projects (Log of BSCP contribution amounts, 2008).

With program funding received to date, PHAC has:

• Established an operational transfusion-related adverse event surveillance system in all but two of the provinces and territories;
• Established  an operational national transfusion-related errors surveillance system;
• Begun planning and consultation for the development of a national surveillance system for transplantation of tissues and organs;
• Begun to establish linkages with public health networks at the province/territory level;
• Established a blood sample archive for high risk populations;
• Begun to analyse and report the data about transfusion-related adverse events and errors;
• Used the TTISS data to identify, assess, and develop mitigating strategies for transfusion-related risks, including bacterial contamination and TACO/TRALI;
• Contributed to patient safety through increased recognition of adverse events in hospitals;
• Established networks both within Canada and internationally; and
• Developed a strong international reputation in blood safety for Canada.

Spending and Efficiencies in Other Jurisdictions

While it was not possible to draw direct comparisons between BSCP spending and the cost of similar programs in other jurisdictions (due to differences in geography, regulation, organization of the systems, etc.), informed supposition and anecdotal information would indicate that similar surveillance systems in other jurisdictions are substantially more costly.

France’s system, for example, according to Faber (2003) is probably costly, as it is extremely resource-intensive. As of 2003, Faber describes the human resources to include: “around 2000 correspondents in hospitals with transfusion activities, hundreds of them being dedicated full-time to this task, and 150 correspondents in blood establishments, 28 regional officers, and staff in the headquarters” (2003, p. 241). However, it is also important to remember that France has a mandatory, rather than voluntary, system.

One expert panellist indicated that the U.S. is currently spending about half the total BSCP budget on a fairly small pilot project. Thus, the Canadian system looked to be “relatively inexpensive by contrast”.

The bottom line is that little information is available about the actual costs associated with different hemovigilance systems (Faber, 2003), and information that is available on the financing of blood safety systems around the world is inadequate (Politis, 2005). There are no existing benchmarks for an acceptable blood safety expenditure (Stainsby, 2006). This makes it difficult to determine cost-effectiveness and value-for-money of hemovigilance systems (Politis, 2005). Politis recommended that the following steps be taken towards understanding spending on a hemovigilance system:

• Priorities should be defined, and decisions made about the data and indicators that must be collected;
• Chain from donor to patient should be analysed to identify opportunities for cost reduction;
• Best practices should be implemented using effective benchmarking procedures; and
• International cooperation should take place to increase awareness of the economics of blood. (2005: 6)

When considering design alternatives, one must contemplate whether differences between systems are meaningful. It is important to note that “it is as yet unclear which type of system is optimal in terms of prevention of serious harm to patients and cost-effectiveness” (Williamson, 2002: 1249). It is difficult to compare systems, as the social and environmental conditions, discussed earlier, are different everywhere. Thus, the comparisons provided do not necessarily determine what the ‘better’ way is.

Perceptions of BSCP Value for Money and Program Efficiency

All of the survey respondents felt that the BSCP budget of $4 million provides good value (42%) or very good value (58%) for taxpayers’ dollars. Similarly, all ten PHAC staff who were interviewed also strongly felt that the BSCP provided either very good or good value.

Interviewees also shared the reasons they felt the program was good value for money. These included:

• A lot is done with a very small amount of funding;
• It increases public confidence;
• It promotes safety;
• It provides information on blood safety within Canada; and
• It provides education on blood safety for front line staff.

 
It is also worth noting that a couple of interviewees cautioned that it will be challenging to implement the CTOSS system within the current BSCP funds. This could have implications for fully implementing CTOSS.

When asked about ideas for increased program efficiency, several interviewees responded that it could not become any more efficient, as it does so much with so few funds already. Others did have suggestions for increased efficiency, including: 

• Review contribution agreement policies and procedures, considering alternatives or ways to reduce delays in renewal (creates inefficiencies);
• Holding TTISS working group meetings differently (e.g., have web-based instead of in-person meetings; consider not having Senior Project Manager from the DGO’s office attend all meetings; and structuring the content of the meetings in a different way by having PHAC integrate provincial/territorial data in order to have more up-to-date information); and
• Hire and retain more PHAC staff, so that more program activities can be accomplished.

Summary

In 2007-2008, the annual program budget was approximately $3.92 million, including contributions, staff salaries, and operations costs. As described in the answers to evaluation questions 7, 9, and 10, a considerable amount has been achieved with BSCP program dollars to date.

The cost of administering the program is approximately 40% of the total program cost, based on program budgets. The program does not usually distribute all of the available funds, in part because of delays in signing the contribution agreements and in part because of the PHAC annual budget reduction exercise. The amount varies from year to year, from 63% of the approved funds being distributed (in 2006-07) to 87% (in 2007-08).

Although it is not possible to draw any definitive conclusions about the efficiency of Canada’s system compared with those in other countries, supposition and anecdotal information indicate that Canada has achieved an internationally recognized system with relatively few funding dollars. Stakeholders also indicated that the BSCP provided good value for money. Very few suggestions for improving efficiency were provided.

Footnotes