Audit of the Biologics and Radiopharmaceutical Drugs Program at Health Canada 2018

Final report presented to the Departmental Audit Committee June 20, 2018

Table of Contents

• List of Acronyms
• Executive Summary
• A - Introduction
  • Background
• B - Findings, Recommendations and Management Responses
  • Governance - Oversight Mechanisms
  • Risk Management
  • Internal Controls
    • Pre-Market Activities - Clinical Trial Applications and Clinical Trial Application Amendments
    • Pre-Market Activities – New Drug Submissions and Supplemental New Drug Submissions
  • Internal Controls
    • Adverse Reaction Monitoring and Assessment
    • Post-Market Activities - Signal Detection
  • Internal Controls
    • Program Operational Reporting
• Conclusion
• Appendix A – Scorecard
• Appendix B – About the Audit
  1. Audit Objective
  2. Audit Scope
  3. Audit Approach
• Appendix C- Lines of Enquiry and Criteria
• Appendix D – Roles and Responsibilities

List of Acronyms

AR

Adverse Reaction

ARIS

Adverse Reaction Information System

BGTD

Biologics and Genetic Therapies Directorate

BOP

Branch Operational Plan

BRDP

Biologics and Radiopharmaceutical Drugs Program

BRR

Branch Risk Register

CTA

Clinical Trial Application

CTA-A

Clinical Trial Application Amendments

DMC

Directorate Management Committee

HPFB

Health Products and Food Branch

IDMC

Independent Data Monitoring Committees

ISO 

International Organization for Standardization

MAH

Market Authorization Holder

MHPD

Marketed Health Products Directorate

NDS

New Drug Submission

PSUR

Periodic Safety Update Reports

RORB

Regulatory Operations and Regions Branch

SNDS

Supplemental New Drug Submission

SOP

Standard Operating Procedure

SSR

Summary Safety Review

TPD

Therapeutic Products Directorate

Executive Summary

What we examined

We examined the management control framework related to the Biologics and Radiopharmaceutical Drugs Program (BRDP or the Program).  

The audit focused on governance, risk management, and internal controls, as they relate to:

• pre-market activities, such as the process to screen and review clinical trial applications;
• new drug submissions and supplemental new drug submissions received by Health Canada, as well as on-site evaluations;
• post-market activities, such as processes related to Adverse Reaction (AR) reporting and signal detection;
• processes related to operational reporting (dashboards) for the Program; and
• Program-related interactions between the Health Products and Food Branch (HPFB) and the Regulatory Operations and Regions Branch (RORB).

Why is it important

Health Canada (HC or the Department) verifies that regulatory requirements for the safety, quality, and efficacy of biologics and radiopharmaceuticals are met through risk assessments, monitoring and surveillance, and compliance and enforcement activities. In addition, the Program provides information to Canadians and key stakeholders, including physicians and pharmacists, to enable them to make informed decisions about the use of biologics and radiopharmaceuticals.

What we found

We found that an adequate management control framework was in place, including governance, risk management practices, and internal controls to support program management for both pre- and post-market activities. Management processes and practices were effective in the following areas:

• comprehensive committee governance structures that support effective oversight, intra- and inter-branch communication, risk awareness and management, and operational cooperation;
• risk management practices that were effectively embedded in the operational activities related to the Program;
• an International Organization for Standardization (ISO) certified Quality Management System, along with dedicated resources and comprehensive standard operating procedures (SOPs) to guide, support, and monitor the performance of key program activities;
• an effective system of operational reporting, with an emphasis on meeting established standards for review and assessment activities; and
• activities in support of stakeholder engagement, continuous learning, and improved processes.

We also identified opportunities for improvements in the processes for review and assessment of ARs, specifically the need to:

• re-assess the current strategy and associated risks related to the review and assessment of post-market AR reports;
• develop a strategy and related processes for the consistent undertaking of branch activities related to AR occurring during clinical trials; and
• implement a more effective and efficient system in support of signal tracking throughout the signal lifecycle.

The areas for improvement noted in this audit report and associated recommendations will collectively strengthen the management of the BRDP. Issues identified as being minor in nature have been brought to management’s attention.

A - Introduction

1. The appendices for this audit report provide additional information on the results of the audit and how it was conducted: Appendix A – Scorecard, Appendix B – About the Audit, and Appendix C – Lines of Enquiry and Criteria.

Background

2. Health Canada (HC or the Department) is the sole body in Canada responsible for the regulation of biologics and radiopharmaceutical health products^{Footnote 1}. The Department of Health Act, the Food and Drugs Act and the Food and Drugs Regulations, the Medical Devices Regulations (with respect to combination products), the Services Fees Act, the Safety of Human Cells, Tissues and Organs for Transplantation Regulations, the Processing and Distribution of Semen for Assisted Conception Regulations, the Blood Regulations, and the Fees in Respect of Drugs and Medical Devices Regulations (with respect to combination products) provide the regulatory framework to develop, maintain, and implement the Biologics and Radiopharmaceutical Drugs Program (BRDP or the Program).
3. The Program’s objective is to ensure that biologics and radiopharmaceuticals in Canada are safe, effective, and of high quality. Actual spending by the Program in 2016-17 was $51,739,190, with 436 full-time equivalents.
4. Products regulated by the Program include drugs produced via recombinant Deoxyribonucleic Acid (DNA) technology^{Footnote 2}, blood and blood products, viral and bacterial vaccines, gene therapy products, tissues, organs, and xenografts^{Footnote 3}. These may be manufactured in Canada or elsewhere.
5. The Department plays a key role in the pre- and post-market activities that contribute to the regulatory framework of the Program, namely the pre-market review and approval of: Clinical Trial Applications (CTAs), New Drug Submissions (NDS), Supplemental New Drug Submissions (SNDS), On-Site Evaluations (OSEs), and the labeling review process. The post-market activities within the regulatory framework include the detection of safety issues, such as Adverse Reaction (AR) monitoring, health risk monitoring, and other surveillance activities.
6. HC verifies that regulatory requirements for the safety, quality, and efficacy of biologics and radiopharmaceuticals are met through risk assessments, monitoring and surveillance, and compliance and enforcement activities. In addition, the Program provides information to Canadians and key stakeholders, including health professionals such as physicians and pharmacists, to enable them to make informed decisions about the use and management of biologics and radiopharmaceuticals.
7. The Program is managed through the Health Products and Food Branch (HPFB or the Branch). The Regulatory Operations and Regions Branch (RORB) also plays a key role in the Program’s delivery (see Appendix D for a detailed breakdown of roles and responsibilities).

B - Findings, Recommendations and Management Responses

Governance - Oversight Mechanisms

8. A governance framework is a set of rules and practices by which an organization ensures accountability, fairness and transparency for all its stakeholders. Ultimately, the application of good governance contributes to the effective and efficient realization of strategic and organizational goals.
9. We expected to find governance mechanisms with clearly defined roles, responsibilities, membership, and decision-making authority that support the strategic direction of the Program.
10. We found that senior management committees were in place at both the branch and directorate levels. In addition, there were various operational-level committees and working groups. This governance structure reflects the complexity and shared responsibilities of the Program’s activities. Through their respective mandates, senior management committees provided an effective oversight and decision-making function, while supporting collaboration, information sharing, and transparency for the delivery of the Program. Operational committees below the senior management level provided additional support and direction for the Program.
11. We reviewed a sample of 15 Terms of Reference (ToRs) for senior- and working-level committees and found that 11 ToRs were not current and another two committees did not have ToRs. We noted that changes were made to the frequency of meetings, membership, and the governance models of the Branch Executive Committee (BEC) and the BEC-IPPS (Integrating Policy, Programs and Science) that were not reflected in an updated ToR. We also noted that multiple committees were not meeting as frequently as expected.
12. Updated ToRs for all key committees would ensure accountabilities are appropriate, optimized and documented. ToRs will also help ensure committees are focused on their areas of responsibility and that the memberships are appropriate for the assigned accountabilities.
13. We also found that, in support of effective governance, RORB (the branch responsible for compliance and enforcement activities) was involved in establishing the formal governance structure through membership on multiple committees and ongoing communication with HPFB. Communication included sharing of risk-related information, results from complete compliance and enforcement procedures (including good manufacturing practices, site inspections, on-site evaluations, signal detection, and AR reports), and results of pre-and post-market activities.
14. Overall, with minor exceptions related to updated documentation of the ToRs of certain committees, we concluded that there were adequate governance mechanisms in place, with clearly defined roles, responsibilities, membership and decision-making authority in support of the strategic and operational direction and oversight function of the Program.

Risk Management

15. Risk management is an integral component of good management that supports organizations in making informed decisions for allocating resources, mitigating threats, and proactively capitalizing on opportunities. However, risk management is not a “one size fits all” approach, but rather a process that is tailored to the organizational environment, including its mandate, structure, operations, and related constraints.
16. We expected to find an established risk management process for the Program where risk activities were identified, assessed, managed, and reported effectively.
17. We found that the organization and documentation of the Branch’s risk management approach and related activities were not specific to any single program, including that of the BRDP. Similarly, key initiatives and activities undertaken were organized at the branch or directorate level, rather than by individual program. However, we found that the risk management process in place within the Branch incorporates and addresses key risks related to the BRDP, and was effectively integrated within the Departmental Integrated Operational Planning Process through the Branch Risk Register (BRR) and the Branch Operational Plan (BOP).
18. The BRR identified key risks that were also reflected in the Corporate Risk Profile (CRP). Branch directorates review and provide input into the BRR on an annual basis. BOPs reflect ministerial as well as branch priorities that were aligned with the five-year Branch Strategic Plan, including initiatives and activities that relate to the Program.
19. A mid-year review of the CRP and an update of the Corporate Overview of Operational Plans process allowed for the monitoring and review of branch priorities represented in the BRR and BOP, at the branch and corporate levels. Non-priority initiatives were monitored and reported both formally, through documented reporting, and informally through bilateral discussions. Directorate Management Committees (DMCs) within each directorate were actively engaged in discussing and assessing risk issues, as well as progress on activities and initiatives.
20. We identified key initiatives and activities that collectively address the risks and strategic priorities for all HPFB’s sub-programs, including risks associated with the BRDP. Such initiatives and activities included:
    • Regulatory openness and transparency;
    • Plain language labelling;
    • Strengthening international regulatory collaboration and cooperation;
    • Increasing effectiveness of the regulatory framework for radiopharmaceuticals;
    • Data mining as a means to enhance the Branch’s effectiveness in accessing global ARs and signals; and
    • Facilitation of screening and review processes through the development of a screening and review tool by the Biologics and Genetic Therapies Directorate (BGTD).
21. In addition, HPFB embedded risk management into its operational structure through ongoing processes and activities to create a cohesive integrated risk management environment. Key processes and activities in support of this include:
    • A quality management system (International Organization for Standardization (ISO) 9001:2008 certified) as a control to ensure standardization and consistency in procedural activities;
    • Systematic and ad hoc activities, such as lessons learned and ‘post mortem’ discussions by working groups and committees including the DMCs, consideration of prior audit recommendations, a process for continuous learning and improvement as part of the ISO quality management system in place, and solicitation and communication of stakeholder and employee feedback;
    • Joint participation at various committees by Branch directorates and bureaus addressing health risks and proposed actions; and
    • Pre- and post-market assessment and surveillance activities and related communication strategies including defined procedures for informing Market Authorization Holders (MAHs) and communicating health risks to the public.
22. Formalization of risk management processes at the directorate level was evolving and not yet standardized. However, the following processes were being proactively reviewed and re-assessed for the purpose of continuous improvement:
    • Risk session presentations have been made to DMCs and to the Planning and Performance Measurement Working Group (PPMWG);
    • Discussions led by PPMWG have taken place to review the BRR process including the consideration for more emphasis on program-level risk management; and
    • Directorate participation in ‘regulatory risk review workshops’ including presentation of results to directorate and branch management committees as well as a “Risk Summit” discussion involving senior branch and HC management.
23. We concluded that, overall, HPFB has implemented adequate processes for identifying, assessing, managing, and reporting on key risks and related initiatives, including those associated with the Program. Furthermore, the Branch was taking proactive steps to review its risk management processes, including the feasibility of further integration and formalization of program- and directorate-level risk factors and associated activities with existing processes at the Branch and corporate level.

Internal Controls

Pre-Market Activities - Clinical Trial Applications and Clinical Trial Application Amendments

24. A clinical trial is an investigation of a drug for use in humans that involves human subjects, and is intended to discover, ascertain or verify the efficacy and safety of the drug prior to its manufacture and marketing. Sponsors of clinical trials must submit Clinical Trial Applications (CTAs) and CTA – Amendments (CTA-As) to the Branch for review and acceptance prior to the conduct of clinical trials.
25. We expected to find established and effective processes for the screening and review of clinical trial applications related to biologic and radiopharmaceutical drugs.
26. We found that the Program communicated the requirements of clinical trial submissions to industry and provided direction by posting guidelines on the Health Canada website. Within the Branch, there is an established process, including Standard Operating Procedures (SOPs) for the screening and review of clinical trials.
27. We tested a sample of 45 CTA and CTA-A files. Overall, in the sample of files reviewed, the CTAs and CTA-As were screened and reviewed according to the established SOPs, within the established timeframes, and approvals were properly documented. However, we found 15 exceptions related to the review stage of the submission process. All the exceptions related to administrative requirements and included: review reports dated later than the final decision letter, review reports not being uploaded to the official document repository system (docuBridge), and a Review Report not being signed.
28. The potential consequences of such errors are unlikely to pose any significant health risks. However, gaps in adherence to administrative requirements, including the need for signatures and dates to be accurately recorded, and for documents to be saved and uploaded, may result in challenges in the efficient retrieval of up-to-date information and effective demonstration of due diligence exercised in the review and approval of CTAs and CTA-As.
29. Notwithstanding the administrative exceptions noted above where minor improvement is required, we concluded that, overall, there were adequate processes in place for the timely screening and review of clinical trials related to biologic and radiopharmaceutical drugs.

Pre-Market Activities – New Drug Submissions and Supplemental New Drug Submissions

30. New Drug Submissions (NDS) and Supplemental New Drug Submissions (SNDSs) are sent to Health Canada by a sponsor (e.g., drug company) for the purpose of getting a new drug approved and available for sale in Canada. The aim of Health Canada is to authorize safe, effective, and high-quality drugs. The drug review process ensures submissions were reviewed and approved by scientists at Health Canada.
31. We expected to find established and effectively implemented processes in place for screening and reviewing new drug and supplemental drug submissions.
32. Guidelines providing direction to sponsors for completing their submission package are posted on Health Canada’s website. Established processes, including detailed SOPs, are also in place for the screening, review, and approval of NDSs and SNDSs by Branch staff.
33. We tested a sample of 27 NDS and SNDS files to confirm that screening and review of the applications submitted were completed as per the established SOPs and included the required approvals. We found 11 exceptions that were administrative in nature which included: review reports not being dated, documents not being uploaded to the data repository system (docuBridge), and reports not being signed.
34. Overall, with the exception of needing minor administrative improvements, there were adequate processes in place for the screening and review of new drug and supplemental new drug submissions related to biologic and radiopharmaceutical drugs.

Internal Controls

Adverse Reaction Monitoring and Assessment

35. Adverse reactions (ARs) are undesirable effects from health products, including biologic and radiopharmaceutical drugs. They can arise during clinical trials, after biologics and radiopharmaceuticals have been marketed (post-market), or are in use (donating blood and receiving blood and blood products). The Branch has different requirements in place for the review and assessment of ARs on the basis of when and how the ARs arise, the suspected causes, and the nature of the reaction. To best reflect the above differences, our findings below are presented under the categories of a) post-market ARs (non-blood and non-blood products), b) ARs to blood and blood products and c) ARs during clinical trials.
36. We expected to find established and effective processes for the identification, coding, assessment, tracking, and follow-up on ARs related to biologics and radiopharmaceutical drugs.
37. We found established processes for the identification, coding, tracking, and following-up on AR reports for authorized products related to biologics and radiopharmaceutical drugs, with some exceptions noted.

Post-Market Adverse Reactions (non-Blood and non-Blood Products):

38. ARs may originate both within and outside Canada, the latter being referred to as “foreign reports”. The majority of AR reports are foreign reports. Branch management stated that it is not feasible to assess foreign reports, given the large number received annually (more than 700,000, compared to approximately 120,000 domestic). Hence, only domestic AR reports are reviewed and assessed. However, the Branch has identified a data mining project under development to facilitate the consistent, effective, and efficient screening of both domestic and foreign AR reports. If this project is successful, the Branch will be able to systematically screen AR data to provide more comprehensive monitoring of marketed Canadian health products.
39. We found that processes for the receipt, coding, screening, and recording of AR reports were documented within various SOPs and were being consistently followed. This included the use of a dedicated system, the Adverse Reaction Information System (ARIS) database, where ARs and associated minimum required data were recorded. We also found that a quality assurance process was in place to monitor and assess the documentation and coding of mandatory fields in the ARIS database.
40. The recording of ARs was identified and associated with the individual experiencing the AR. There may be ARs reported after the initial report (version zero), which are recorded in subsequent versions.
41. Review and assessment of domestic AR reports were risk-based, pursuant to the Branch’s Targeted Monitoring Strategy (TMS) that focused on ARs that the Branch deems to be more serious. Such reactions included, but were not limited to, reactions with a chance of a fatal or life-threatening outcome, those in the pediatric population, those related to New Active Substances, or those that were rare with a high drug-attributable risk.
42. We examined a sample of 50 domestic reports that should have been reviewed according to the TMS. We found that five of these reports were not assessed. Management stated that, even where reactions fall under the TMS (with the exception of cases where the outcome is fatal or life-threatening), only the initial AR report was assessed. Subsequent ARs related to the same subject were not assessed.
43. Based on these testing results, there was significant risk that the current assessment strategy may result in ARs with potential significant health consequences not being assessed.
44. We found that the remaining 45 ARs examined were all assessed according to the established process. However, we found that there was no quality assurance or peer review function in place to challenge or confirm the decisions of the initial reviewer. We noted that peer reviews were undertaken when the AR reviews were conducted by newly trained staff, and in the relatively few instances where the AR was referred to the Signal Detection Meeting that included peers and subject matter experts.
45. Peer review and confirmation of the initial assessment decision would enhance the review and assessment processes by minimizing the risk of error or misjudgement in initial AR assessments.

Recommendation 1

Management response

Adverse Reactions during the Clinical Trial Process:

46. Adverse reactions (ARs) can also occur in the pre-market phase affecting clinical trial participants.
47. The Branch does not request or review ARs arising during clinical trials. The duty to investigate the ARs was placed on the sponsor. We were told that, although not mandated through regulations, sponsors put in place Independent Data Monitoring Committees (IDMCs), whose role was to monitor the progress and results of the clinical trials, including the assessment of ARs. The IDMCs inform the decision of whether or not to proceed with the trials.
48. There was no established process to systemically request, review, and assess clinical trial AR reports or related reports and decisions of the IDMCs. Management stated that this has been identified as an area requiring attention, and intends to develop standard operating procedures to address it.
49. A systematic and consistent process for responses to ARs arising during clinical trials would enhance the Branch’s ability to demonstrate that it is exercising sufficient review over potential risks to clinical trial participants.

Recommendation 2

Management response

Adverse Reactions Related to Blood and Blood Products:

50. Pursuant to the Blood Regulations, establishments are required to provide notice to the Minister of ARs experienced by those who donate or receive blood. The vast majority of reported ARs relate to blood recipients. The Branch assigns a high priority to reviewing and assessing ARs arising from blood and blood products.
51. In the case of blood donors, notification must be provided of where the AR took place, during or within 72 hours after the donation. For blood recipients, notification to the Minister is required when establishments conduct an investigation into the AR. Investigations must be conducted in cases where the AR is serious or unexpected, and where there is an indication that the root cause of the AR is attributable to the establishment’s activities. The need for an investigation is at the discretion of the blood establishments. Where investigations are conducted by blood establishments, they must file a final report with the Minister outlining the results, the final disposition of the blood that was the subject of the investigation, and any relevant corrective actions taken.

Assessments of Adverse Reaction Reports:

52. We found that the Branch assigns a high priority to reviewing and assessing AR reports arising from blood and blood products, due to the increased potential risk to health associated with blood and blood products. We also found that establishments notified the Minister of ARs as a matter of course, even in the majority of cases where the reaction was not deemed ‘reportable’, pursuant to the Blood Regulations (i.e., there was no indication that the AR was caused by the blood or blood product, or the associated activities of the establishment). In turn, the Branch reviewed, assessed, and tracked all reports received, pursuant to established SOPs and timelines. According to management, blood establishments, including blood operators, hospitals and clinics, were in the best position to assess ARs and determine if the reaction was caused by a blood donation or transfusion, or other associated activities of the establishment. This was supported by the fact that the Blood Regulations themselves assign responsibility to the establishments for making these determinations including the need to undertake an investigation.
53. This Branch review process, although based on a determination of high risk associated with blood and blood product ARs, resulted in significant duplication of effort to confirm or challenge the decisions of blood establishments. This is emphasized by the fact that the establishments were in a better position, and were charged with the responsibility, to assess ARs and determine the likelihood that the AR was caused by the blood or the associated blood donation or blood transfusion activities of the establishment.

Annual Reports from Blood Establishments:

54. The Blood Regulations require blood establishments (hospitals and clinics), which include Canadian Blood Services, Héma Quebec, and 180 blood banks across Canada, to prepare annual reports summarizing the individual investigations undertaken related to ARs of blood donors or recipients. These reports are required to be submitted to the Minister upon request. We found that, during the scope of the audit, such annual reports had not been requested of the blood establishments by the Branch.
55. The request and review of such reports, even on a select risk basis, would supplement and enhance the effectiveness and efficiency of reviewing and assessing risks related to ARs associated with blood and blood products.

Branch Annual Reports on Blood and Blood Products Adverse Reactions:

56. In 2015, a Memorandum of Understanding between directorates within the Branch delineated responsibilities related to ARs arising from blood and blood products. The assigned responsibilities included the preparation of an annual report on blood and blood component AR reports that were received and evaluated by the Branch during the year. We found that a draft annual report was prepared for calendar year 2015 but never finalized. No other such report has subsequently been prepared. The Branch attributed this to resource constraints and competing priorities.
57. The compilation of such a report may be a valuable tool to management in support of the development and update of policy, regulations and operational procedures related to ARs associated with blood and blood products, and the associated allocation of resources.
58. We concluded that, although there were established processes for the identification, coding, tracking, and following-up of AR reports related to biologics and radiopharmaceutical drugs, there were areas that require improvement.
59. Recommendations 1 and 2 in this section of the report outline actions for improvement in these areas.

Post-Market Activities - Signal Detection

60. Signals are information reports on a possible causal relationship between an adverse event (any unfavorable or unintended medical occurrence in a patient or clinical investigation) and a drug.
61. We expected to find established processes to identify, prioritize, assess, track, and follow-up on signals related to biologic and radiopharmaceutical drugs.
62. We found a documented post-market drug safety surveillance process was in place for the identification, prioritization and assessment of potential signals related to biologic and radiopharmaceutical drugs.

Signal identification, prioritization, and assessment:

63. We found that there was a systemic process in place to identify signals. This involved the scanning of multiple sources of information, including scientific literature, regulatory agencies’ communications and Periodic Safety Update Reports (PSURs) submitted by MAHs. Once signals were identified, the signal, along with related information gathered, was reviewed by the Signal Prioritization working group to determine whether it was to be dismissed, researched further, or prioritized for signal assessments.
64. We tested a sample of 40 signal files and found that they were properly identified, prioritized and assessed, according to the established processes and related SOPs.

Signal tracking, follow-up and reporting:

65. Multiple manual tracking sheets were maintained by the Branch to ensure potential signals were appropriately prioritized and assessed, and that follow-up actions were tracked.  However, there was no mechanism for notification, nor a reminder system to follow up on signals. The Branch was working on implementing a common tracking system (CTS) to replace the current system in use. The Branch was unsure of the capacity of a CTS to support the signal lifecycle activities and replace all current tracking spreadsheets also in use.
66. We found that for certain signal assessments, HC prepares and publishes Summary Safety Reviews (SSRs) as part of its ongoing commitment to openness and transparency.  These reports complement other safety-related information and helps Canadians make informed decisions about their medication choices. An SSR is prepared when there is no imminent health and safety risk to the public. Each SSR outlines what was assessed, what was found, and what action, if any, was taken by Health Canada.
67. In the sample tested, SSRs were required for three signal assessment files. We found that these three SSRs were posted on Health Canada’s Website between 171 and 279 days after the completion of the assessment. Follow-up with management confirmed that established performance standards for the publication of SSRs are in place; however, the sample tested did not meet those standards. 
68. Although there were processes in place for the communication of health risks to partners, stakeholders and the public through other more proactive and timely means, communication of the Department’s actions through timely publication of the SSRs would better support the Department’s mandate for openness and transparency.
69. Overall, we found that there were established processes to identify, prioritize, and assess signals related to biologic and radiopharmaceutical drugs. However, the manual nature of systems presents challenges to the effective tracking and follow-up of signals, requiring that moderate improvements be made.  

Recommendation 3

Management response

Internal Controls

Program Operational Reporting

70. Operational reporting provides senior management with program information to support decision making.
71. We expected to find processes in place to capture and review operational results for accuracy and completeness, and to provide senior management with information for making decisions related to the Program.
72. We found that there is an operational dashboard process in place to inform senior management. Data sources and methods to extract information for the creation of operational reports and resulting dashboards were established and being consistently used. However, there was some duplication of effort in the review and preparation of the reports and dashboards, namely for dashboards related to signal detection related activities.
73. The responsible individuals were aware of their roles and responsibilities for the collection and recording of information, the creation of reports, and for reviewing the accuracy and completeness of the reports. The relevant Directors and Directors General were involved in reviewing the reports and approving them prior to being presented to the Assistant Deputy Minister’s office.
74. Given that the same individuals have been involved in the role of creating the dashboards, the procedural steps were known to them. However, the procedures were not fully documented or updated. Due to the absence of a standard system, there continues to be extensive use of Excel spreadsheets for the creation of dashboards and reports. The use of multiple tracking spreadsheets and manual input creates some duplication of work and increases the risk of error due to data entry or loss of information.
75. We found processes in place to capture and review operational results for accuracy and completeness, and provide senior management with information for decision making related to the Program.

Conclusion

76. We concluded that, overall, there was an adequate management control framework in place including governance, risk management, and internal control structures, to effectively support pre- and post-market activities of the Program.
77. Areas where good practices were identified include:
    • comprehensive committee governance structures that support effective oversight, intra- and inter-branch communications, risk awareness and management, and operational cooperation;
    • risk management practices that were effectively embedded in operational activities related to the Program;
    • an ISO certified Quality Management System, along with dedicated resources and comprehensive SOPs to guide, support, and monitor the performance of key program activities;
    • an effective system of operational reporting with an emphasis on meeting established standards for review and assessment activities; and
    • activities in support of stakeholder engagement, continuous learning, and improved processes.
78. We also identified opportunities for improvements in the processes for review and assessment of ARs, specifically the need to:
    • re-assess the current strategy and associated risks related to the review and assessment of post-market AR reports, along with implementation of a peer review function related to AR assessment decisions;
    • develop a strategy and related processes for consistent Branch activities related to ARs arising during clinical trials; and
    • implement a more effective and efficient system in support of signal tracking throughout the signal lifecycle.
79. The areas for improvement noted in this audit report will collectively strengthen management of the Biologics and Radiopharmaceutical Drugs Program. Issues identified as being minor in nature have been brought to management’s attention.

Appendix A – Scorecard

Audit of Biologics and Radiopharmaceutical Drugs Program at Health Canada

Criterion	Rating	Conclusion	Rec #
Governance
1. Oversight Mechanisms	No deficiencies	1.1 Oversight mechanisms with clearly defined roles, responsibilities, membership, and decision-making authority for the Biologics and Radiopharmaceutical Drugs Program were effectively implemented. Sharing of information from pre-market and post-market activities, as well as, compliance and enforcement activities takes place between HPFB and RORB for the Program.	-
Risk Management
2. Risk Management	No deficiencies	2.1 A risk management process for the Biologics and Radiopharmaceutical Drugs Program was established where risk activities were identified, assessed, managed, and reported effectively.	-
Internal Controls
3. Pre-market activities	No deficiencies	3.1 Processes for the timely screening and review of clinical trials related to biologic and radiopharmaceutical drugs were established and being used effectively, including approvals and decisions made by authorized individuals.	-
	No deficiencies	3.2 Processes for screening and review of New Drug and Supplemental Drug Submissions related to biologic and radiopharmaceutical drugs were established and were being used effectively, including approvals and decisions made by authorized individuals.	-
4. Post-Market Activities	More than one major deficiency	4.1 Processes were established for the identification, coding, tracking, and following-up on Adverse Reaction reports for authorized products related to biologics and radiopharmaceutical drugs. However, there was significant risk that the current assessment strategy may result in Adverse Reactions with potential significant health consequences not being assessed. In addition, there was no systematic and consistent process in place to deal with Adverse Reactions arising during clinical trials. This may result in the Branch’s inability to demonstrate that it is adequately assessing potential health and safety risks to clinical trial participants.	R-1, R-2,
	One major deficiency	4.2 Processes were established to identify, prioritize, and assess signals related to biologic and radiopharmaceutical drugs. However, challenges exist for the effective and efficient tracking and follow-up of signals due to the manual nature of systems.	R-3,
5. Program Operational Reporting	No deficiencies	5.1 Processes for reporting related to the Biologics and Radiopharmaceutical Drugs Program were established to inform senior management on operations.	-

Appendix B – About the Audit

1. Audit Objective

The objective of this audit was to assess the Management Control Framework for the Biologics and Radiopharmaceutical Drugs Program.

2. Audit Scope

The scope of this audit included pre-market and post-market processes and activities in place from April 1, 2015, to January 31, 2018, as well as the interactions between the Health Products and Food Branch and the Regulatory Operations and Regions Branch related to the Biologics and Radiopharmaceutical Drugs Program. The areas of focus included:

• Pre-market activities: screening and review processes related to Clinical Trial Applications, New Drug Submissions and Supplemental Drug Submissions; and
• Post-market activities: post-market activities related to Adverse Reaction (AR) reporting and signal detection.

Activities not in Scope

In June 2015, the Internal Audit team conducted an Audit of Cost Recovery of Health Products that included the Biologics and Radiopharmaceutical Drugs Program. The audit made six recommendations, four of which remain open. Cost recovery was therefore out of scope for this audit because of the ongoing implementation of previous audit’s recommendations.

Compliance and enforcement was out of scope for this audit because Internal Audit is planning a horizontal audit on the Inspection Framework in 2018-19. This audit will include compliance and enforcement for all regulatory programs, including the Biologics and Radiopharmaceutical Drugs Program,  as well as site inspections of manufacturing facilities. The audit will complement the work completed for the Comprehensive Review.

The recall process was out of scope for this audit because Internal Audit is planning a horizontal Audit of Recall Practices in 2019-20. This audit will include alerts or advisories triggered by industry or surveillance activities and signal assessments. This audit will round out our examination of the post-market regulatory activities.

Furthermore, international collaboration, including information shared between Health Canada and other regulators, such as the United States Food and Drug Administration, the Australian Therapeutic Goods Administration, and the European Medicines Agency, as well as the Public Health Agency of Canada, was not included within the scope of this audit. International collaboration was not included, as it is not solely under the control of Health Canada.

3. Audit Approach

We conducted the audit in conformance with the Treasury Board of Canada’s Policy on Internal Audit. The audit was conducted at Health Canada’s headquarters. The principal audit procedures included, but were not limited to:

• A review and analysis of policy frameworks, planning documents, service delivery and performance related documentation;
• Walkthroughs and interviews with key program personnel at headquarters;
• Observation, inquiry, and testing of related controls over pre-market and post-market activities; and
• A sampling strategy aimed at ensuring that sufficient evidence was obtained for the criteria being addressed through the detailed testing approach.

Statement of Conformance

In the professional judgment of the Chief Audit Executive, sufficient and appropriate procedures were performed and evidence gathered to support the accuracy of the audit conclusion. The audit findings and conclusion are based on a comparison of the conditions that existed as of the date of the audit, against established criteria that were agreed upon with management. Furthermore, the evidence was gathered in accordance with the Internal Auditing Standards for the Government of Canada and the International Standards for the Professional Practice of Internal Auditing. The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.

Appendix C- Lines of Enquiry and Criteria

Audit of Biologics and Radiopharmaceutical Drugs Program at Health Canada

Criteria Title	Audit Criteria
Line of Enquiry 1: Governance
1. Oversight Mechanisms	1.1 The Biologics and Radiopharmaceutical Drugs Program has governance mechanisms with clearly defined roles, responsibilities, membership, and decision-making authority that support its strategic direction.
Line of Enquiry 2: Risk Management
2. Risk Management	2.1 A risk management process for the Biologics and Radiopharmaceutical Drugs Program is established where risk activities are identified, assessed, managed, and reported effectively.
Line of Enquiry 3: Internal Controls
3. Pre-Market Activities	3.1 There are effective processes in place for the for the screening and review of clinical trials related to biologic and radiopharmaceutical drugs.
	3.2 There are effective processes in place for screening and reviewing New Drug and Supplemental Drug Submissions related to biologic and radiopharmaceutical drugs.
4. Post-Market Activities	4.1 There are effective processes in place for the identification, coding, tracking and following-up on Adverse Reaction reports for authorized products related to biologics and radiopharmaceutical drugs.
	4.2 Processes are established and operating effectively for identifying, assessing, prioritizing, tracking and following- up on signals related to biologics and radiopharmaceutical drugs.
5. Program Management	5.1 There is an effective process in place for operational reporting related to the Biologics and Radiopharmaceutical Drugs Program.

Appendix D – Roles and Responsibilities

In the context of biologic and radiopharmaceutical drugs, HPFB helps Canadians maintain and improve their health through the following activities:

• evaluating and monitoring the safety, quality and efficacy of biologic and genetic therapies;
• providing timely, evidence-based, and authoritative information to allow healthy and informed decisions for Canadians; and
• anticipating and responding to public health and safety issues associated with health products.

The main directorate involved in delivering the Program is the Biologics and Genetic Therapies Directorate (BGTD). BGTD is responsible for regulating biologics and radiopharmaceuticals for human use in Canada, whether manufactured in Canada or elsewhere. BGTD is responsible for developing new regulations, policies, guidance, and SOPs. It maintains and updates existing documents, conducts pre-market reviews of product submissions to determine safety, efficacy, and quality of biologic and radiopharmaceutical drugs, and the appropriateness of their labelling. BGTD also authorizes the sale of these drugs in Canada, reviews Clinical Trial Applications (CTAs), and ensures that clinical trials are properly designed and do not pose undue risks to participants. It conducts surveillance of adverse reactions (ARs) in clinical trials, including the detection of safety issues, such as AR monitoring, health risk monitoring and surveillance activities,  complaints, and reports of problems (including performing Health Risk Assessments) in support of post-market activities of the Marketed Health Products Directorate (MHPD).

The Therapeutic Products Directorate (TPD) is the federal authority that regulates pharmaceutical drugs and medical devices for human use under the authority of the Food and Drugs Act and Regulations. As a partner in the Program, TPD, through the Office of Submissions and Intellectual Property (OSIP), receives and processes biologic drug submissions, including Drug Identification Number (DIN) issuance and cancellation, and market notification, administers cost recovery activities, administers provisions related to patents and data protection, maintains the Drug Submission Tracking System, Drug Product Database, Notice of Compliance (NOC) database and the docuBridge system, and produces quarterly and annual biologic drug submission reports on behalf of BGTD. TPD also prepares Summary Basis of Decision (SBD) documents on behalf of BGTD for approved new drug submissions for new active substances and biosimilars. TPD publishes information on BGTD submissions in its Submission Under Review Lists, and provides oversight on the implementation of the Regulatory Decision Summary initiative. In addition, TPD also administers the Special Access Programme on behalf of BGTD.

Marketed Health Products Directorate (MHPD) leads an evidence-based vigilance program for health products in Canada. MHPD’s responsibilities in relation to biologics and radiopharmaceuticals are primarily  to monitor a variety of sources for safety signals, including PSURs, Periodic Benefit Risk Evaluation Reports (PBRERs), and other information provided by MAHs, as well as international data. MHPD also manages the Canada Vigilance Database, conducts comprehensive safety assessments, monitors regulatory advertising activities, communicates health product risks to the public and health care practitioners, and develops policies and strategies for post-market surveillance.

Regulatory Operations and Regions Branch (RORB) is the operational arm of HC and also operates in the regions. RORB works in conjunction with HPFB to manage the Program. Responsibilities include compliance promotion activities, as well as enforcement of laws and regulations through inspections, investigations, legal action, and evaluations of compliance with standards affecting good pharmacovigilance practices, manufacturing, packaging and labelling, analysis, importing, distributing, and wholesaling of health products.

The Policy, Planning and International Affairs Directorate (PPIAD) provides leadership in developing and advancing HPFB’s policy and international agendas. This includes policy development on horizontal issues, legislative and regulatory modernization, activities to increase Canada’s influence as a global regulator, and science policy integration.

The Resource Management and Operations Directorate (RMOD) provides branch-wide oversight, coordination and guidance on the consistent, efficient and effective management of operations and resources across all programs led by HPFB, including the Biologics and Radiopharmaceutical Drugs Program.

Regulatory Activities

The regulatory process for biologic and radiopharmaceutical drugs consists of two main categories: pre-market activities and post-market activities.

Pre-Market Activities

Before biologic and radiopharmaceutical drugs can enter the Canadian market, their manufacturer must provide significant scientific evidence that the product is safe, effective, and of suitable quality. This is done through the NDS process which includes the results of the pre-clinical and clinical studies, whether done in Canada or elsewhere, details regarding the production of the drug, packaging and labelling, and information regarding therapeutic claims and side effects. For biologic and radiopharmaceutical drugs, the manufacturer must also supply the Product Specific Facility Information that outlines the methods of manufacture in significant detail. An inspection of the manufacturing facility, known as an On-Site Evaluation, is completed. All these tasks are accomplished by BGTD. If there is sufficient evidence to support safety, efficacy and quality claims, the product is issued a NOC by BGTD and a DIN by TPD indicating that it is approved for sale in Canada.

BGTD also undertakes a risk-based lot release program which dictates the particular testing to be done for each biologic or radiopharmaceutical drug prior to its release on the market. With high-risk biologics, each lot is tested before being released for sale in Canada. Moderate-risk biologics and radiopharmaceuticals are periodically tested at the discretion of HC. Manufacturers of low-risk biologic and radiopharmaceutical drugs usually only need to contact HC regarding lots being sold or for providing certification of complete and satisfactory testing.

Post-Market Activities

Information from adverse drug reaction reports, product complaints, product recalls, and withdrawals contribute to the post-market safety profile of the drug product. MHPD reviews the Risk Management Plans (RMP) within and outside of the pre-market submission, including PSURs, PBRERs, and other information provided by the MAHs. These activities provide health care professionals with the information they need to make informed recommendations to Canadians about biologic and radiopharmaceutical products, and provide the information patients need to make informed decisions.

Health Canada monitors biologic adverse events, investigates complaints and problem reports, maintains post-approval surveillance, and manages recalls as required. This work is carried out by MHPD, while recalls are managed by RORB. In collaboration with the Public Health Agency of Canada, HC monitors vaccinations.