Appendix 1: Evaluation of the Blood Safety Contribution Program at the Public Health Agency of Canada 2008–09 to 2013–14 – Logic model

Appendix 1 – Logic Model

Appendix 1: Project component logic model

Text equivalent for Appendix 1 – Logic Model

Blood Safety Contribution Program Logic Model

Inputs: Contribution funding, human resources, facilities/infrastructure, policies/regulations/international collaboration, and surveillance system

All inputs lead to three activities:

  • Systematic data collection; coordination, analysis and interpretation of data; and dissemination of data

All activities lead to the following outputs:

  • Surveillance data, surveillance reports, guidance documents, conference presentations, and stakeholder meeting presentations.

These outputs reach health system stakeholders responsible for monitoring, preventing, and controlling transfusion and transplantation adverse effects.

The logic model identifies the Program's immediate and intermediate outcomes as follows: timely and useful surveillance evidence for health system stakeholders and health system stakeholders utilize surveillance evidence to inform public on health risks and threats, to inform planning, programming and policy which contributes to regulation and legislation development.

Long-term outcome: Improved patient safety

Appendix 2 – Evaluation Description

Evaluation Issues  

The specific evaluation questions used in this evaluation were based on the five core issues prescribed in the Treasury Board of Canada's Policy on Evaluation (2009). These are noted in the table below. Corresponding to each of the core issues, evaluation questions were tailored to the program and guided the evaluation process.

Table 5: Core Evaluation Issues and Questions
Core Issues Evaluation Questions
Relevance
Issue #1:
Continued Need for Program
Assessment of the extent to which the program continues to address a demonstrable need and is responsive to the needs of Canadians
  • Does the BSCP continue to address a health need?
  • Has the need changed over time?
  • Has the program adapted to changing needs?
Issue #2:
Alignment with Government Priorities
Assessment of the linkages between program objectives and (i) federal government priorities and (ii) departmental strategic outcomes
  • Do the objectives and priorities of the BSCP align with current Government of Canada and Public Health Agency of Canada priorities and strategic outcomes?
Issue #3:
Alignment with Federal Roles and Responsibilities
Assessment of the role and responsibilities for the federal government in delivering the program
  • Do BSCP activities align with federal government jurisdictional, mandated and/or legislated roles and responsibilities?
  • How do BSCP activities relate to the activities of other stakeholders?
Performance (effectiveness, economy and efficiency)
Issue #4:
Achievement of Expected Outcomes (Effectiveness)
Assessment of progress toward expected outcomes (incl. immediate, intermediate and long-term outcomes) with reference to performance targets and program reach, program design, including the linkage and contribution of outputs to outcomes
  • Is timely and useful surveillance evidence available for health system stakeholders?
  • Are health system stakeholders using surveillance evidence to inform public on health risk and threats and for  planning, programming, and policy, regulation and legislation development?
  • Is there improved patient safety?
  • What have been the challenges and lessons learned?
Issue #5:
Demonstration of Economy and Efficiency
Assessment of resource utilization in relation to the production of outputs and progress toward expected outcomes
  • Has the BSCP been implemented economically and efficiently relative to the degree of achievement of the expected outcomes?
Data Collection and Analysis Methods  

The scope of the evaluation included an assessment of the relevance and performance of the Agency's biosecurity program between 2008-2009 and 2013-2014.  Evaluators collected and analyzed data from multiple sources.  Sources of information used in this evaluation included:

Literature review:  A review of the literature using GoogleScholar was performed.  Material was also provided by the Program and stakeholders. 

Document and file review: The Program provided financial and nonfinancial administrative records, surveillance data, supporting documents, presentations, relevant articles, meeting schedules and reports.  The Web was used to collect surveillance reports from relevant countries to determine surveillance reporting characteristics (e.g., frequency of reporting, lag time between data collection and reporting, scope of surveillance).

Key informant interviews:  The first wave of interviews took place with Program staff after a preliminary review of documents and literature were conducted.  Findings from these reviews were used to construct the Program questionnaire and focus the interviews.  This phase of interviewing focused mainly on program activities and outputs.  A second wave of interviews was done with stakeholders (e.g., provincial and territorial funded recipients, Health Canada's MHPD and BGTD, CBS and Hema-Québéc and blood researchers).  A total of 27 individuals were interviewed.  The questions posed in these interviews addressed relevance and performance of the Program. 

Data analysis:  A number of data matrices were created to organize and analyse the data by core evaluation issues and questions. Overall, Program outcomes provided the analytical lens.  Central themes of timely surveillance data, utilization and patient safety were used to develop lines of questioning, data extraction and analysis.  Data were analyzed by triangulating information gathered from the different sources and methods listed above.  Contact was maintained with the Program throughout the data collection and analysis process in order to address emerging questions, for data requests and for technical and analytical assistance.

Appendix 3 – International Comparisons

Country Scope (most recent reports) Donor/ Recipient Safety Mandatory/
Voluntary
Reporting
Incidents/ Errors Morbidity/
Mortality
Publication Year Schedule Lag (years)
Australia Transfusion-related adverse events between July 2009 and June 2011 and donation-related adverse events between July 2011 and June 2012. D/R   I/E MT/MR 2013 Annual 1
Canada 2006 - 2010
2004 - 2005 TTISS
2005 - 2007 TESS
R V ?
I
E
MT/MR 2014 +
2008
2012
Sporadic 3 - 7
3
5 - 7
France Rapport d'activité biovigilance 2012 D/R M I/E MT/MR 2013 Annual 1
New Zealand 2007-11 R   I/E MT/MR 2012 Annual 1
United Kingdom 2009-2012, transfusion errors and adverse events (SHOT) D/R V I/E MT/MR 2013 Annual 1
United States of America Fiscal year 2012, Fatalities Reported to Food and Drug Administration (FDA) Following Blood Collection and Transfusion R M I/E MT 2013 Annual 1
Center for Disease Control_ National Health Safety Network
(Blood, tissues and organs)
Under development
R V I/E MT/MB N/A N/A N/A

Data sources:

Australia: National Blood Authority Haemoviligance Advisory Committee. (2013). Australia haemovigilance report, data for 2009-10 and 2010-11. Retrieved from http://www.blood.gov.au/haemovigilance-reporting on February 10, 2014.

France: Creusvaux, H., Rigault, G, & Roche, M. (2012). Biovigilance in organ procurement and transplantation: French experience and results 2007 – 2011. Organs, Tissues and Cells, 15, 179-185.

New Zealand: Dagger, J., Dinesh, D. National Haemovigilance Programme Annual Report 2011. Retrieved from http://www.nzblood.co.nz/content/download/3066/38247/file/Haemovigilance%20Annual%20Report%202011.pdf on February 10, 2014.

United Kingdom: PHB Bolton-Maggs (Ed), D Poles, A Watt, D Thomas and H Cohen on behalf of the Serious Hazards of Transfusion (SHOT). Steering Group. The 2012 Annual SHOT Report (2013). Retrieved from http://www.shotuk.org/shot-reports/report-summary-and-supplement-2012/ on February 10, 2014.

United States: US Food and Drug Administration. (2012). Fatalities reported to FDA following blood collection and transfusion. Retrieved from http://www.fda.gov/BiologicsBloodVaccines/SafetyAvailability/ReportaProblem/TransfusionDonationFatalities/ucm346639.htm on February 10, 2014.

+ Report has not been released at the time of this writing.

Appendix 4 – References

Alexander, P. E., Hyson, C., & Robillard, P. (n.d.). The development of a Canadian cells tissues and organ surveillance system (CTOSS) by the Public Health Agency of Canada. Draft, program document.

Creusvaux, H., Rigault, G, & Roche, M. (2012). Biovigilance in organ procurement and transplantation: French experience and results 2007 – 2011. Organs, Tissues and Cells, 15, 179-185.

Dagger, J., Dinesh, D. National Haemovigilance Programme Annual Report 2011. Retrieved from http://www.nzblood.co.nz/content/download/3066/38247/file/Haemovigilance%20Annual%20Report%202011.pdf on February 10, 2014.

Ditomasso, J., Liu, Y., & Heddle, N. M. (2012). The Canadian transfusion system: What is it and how can the data be used? Transfusion and Apheresis Science, 46(3).

Dzik, W. H. (2003). Emily Cooley Lecture 2002: Transfusion safety in the hospital. Transfusion, 43(9), pp 1190-1199.

Ganz, P. R. & Wu, J. (2012) Regulatory, Public Health, and International Aspects of Hemovigilance in Canada, in Hemovigilance: An Effective Tool for Improving Transfusion Safety (eds R. R. P. De Vries and J.-C. Faber), Wiley-Blackwell, Oxford, UK.

Heddle, N. M., Liu, Y., Barty, R., Eckert, K., Hsia, C., Tinmouth, A., … Ferguson, D. (2013). Ontario data collection strategy: Proof of concept project – final report. Ontario.

Kleinman, S., Caulfield, T., Chan, P., Davenport, R., McFarland, J., McPhedran, S., … Slinger, P. (2004). Toward an understanding of transfusion-related acute lung injury: Statement of a consensus panel. Transfusion, 44, pg 1774 – 1789.

Lacroix, J., Hébert P. C., Hutchison, J.S., et al. Transfusion strategies for patients in pediatric intensive care units. N Engl J Med 2007;356(16):1609-19.

MacDonald, N. E., O'Brien, S. F., & Delage, G. (2012). Transfusion and risk of infection in Canada: Update 2012. Retrieved from http://www.cps.ca/documents/position/transfusion-and-risk-of-infection-Canada on February 10, 2014.

Maskens, C., Downie, H,. Wendt, A., Lima, A., Merkley, L., Lin,Y., & Callum, J. (2014). Hospital-based transfusion error tracking from 2005 – 2010: Identifying the key errors threatening patient transfusion safety. Transfusion, 54, pp 66 – 73.

National Blood Authority Haemoviligance Advisory Committee. (2013). Australia haemovigilance report, data for 2009-10 and 2010-11. Retrieved from http://www.blood.gov.au/haemovigilance-reporting on February 10, 2014.
Norris, S. (July 10, 2008).  Canada's Blood Supply Ten Years After The Krever Commission.  Library of Parliament. PRB -08-14E.

PHB Bolton-Maggs (Ed), D Poles, A Watt, D Thomas and H Cohen on behalf of the Serious Hazards of Transfusion (SHOT). Steering Group. The 2012 Annual SHOT Report (2013). Retrieved from http://www.shotuk.org/shot-reports/report-summary-and-supplement-2012/ on February 10, 2014.

Public Health Agency of Canada. (2012). Public Health Agency of Canada Surveillance Strategic Plan 2013-2016. Ottawa, ON.

Public Health Agency of Canada. (2012). Surveillance Strategic Plan, 2013-2016, Public Health Agency of Canada, http://intranet.phac-aspc.gc.ca/surveillance/ssp-pss-eng.html.

US Food and Drug Administration. (2012). Fatalities reported to FDA following blood collection and transfusion. Retrieved from http://www.fda.gov/BiologicsBloodVaccines/SafetyAvailability/ReportaProblem/TransfusionDonationFatalities/ucm346639.htm on February 10, 2014. 

Weirsom-Osselton, J. C., Faber, J. C., Politis, C., Brand, A., van der Bom, J. G. & Schipperus, M. R. (2013). Quality validation of data in national haemovigilance systems in Europe: Report of a survey on current state of practice. Vox Sanguinis, 104, 214 – 217.

Report a problem or mistake on this page
Please select all that apply:

Thank you for your help!

You will not receive a reply. For enquiries, contact us.

Date modified: