Study Guide GD211: Guidance on the content of quality management system audit reports
Date Adopted: 2011/11/15
Effective Date: 2011/12/07
This document is a study guide, part of a training programme on the Health Canada guidance document 'GD211: Guidance on the content of quality management system audit report' directed at quality management system auditors performing regulatory audits on behalf of Health Canada and other regulators. This training programme also includes PowerPoint presentation modules as well as an online video training on the United States Food and Drugs Administration's Centre for Devices and Radiological Health (CDRH) Learn portal. Students are encouraged to avail themselves of all available resources on this topic.
This study guide and the associated training programme is not meant to introduce new policy or guidance, and is intended solely to assist auditors and conformity assessment bodies understand and apply the guidance found in the subject document: the limitations presented in the foreword therein apply.
This study guide and its associated training programme were developed in collaboration with the CDRH. The contribution of CDRH is acknowledged and greatly appreciated.
Table of Contents
- 1.0 Introduction
- 2.0 Background
- 3.0 Principles
- 4.0 Users of Reports
- 5.0 Report Format
- 6.0 Report Language
- 7.0 Parts of a Report
- 8.0 Information about the manufacturer
- 9.0 Information about the audit
- 10.0 Audit findings
- 11.0 Conclusions
- 12.0 Identification and dating
- 13.0 Bibliography
This document is a study guide meant to assist conformity assessment bodies (CAB) and their auditors to understand and apply the requirements of GD211. It is part of a larger training programme developed in cooperation with the United States Food and Drug Administration's (FDA) Centre for Devices and Radiological Health, which includes online video training modules on the CDRH Learn Portal and PowerPoint presentations. This training programme does not, in and of itself, contain new requirements. Readers are encouraged to avail themselves of all the elements of this training programme.
This study guide is designed to provide additional insight on the intent and practical application of GD211. While it does contain a number of examples, these are provided for illustrative purposes only. These examples should not be construed as representing specific expectations of audit reporting, nor should they be used as templates.
This study guide also discusses the requirements of the guidance document in depth. It is expected that the insight this will afford auditors will assist them in preparing audit reports that address the requirements. Four general principles are presented to the reader to guide decision making in the absence of specific guidance.
Although GD211 specifies the minimum content of audit reports, it does not constitute a rigid audit report format or template. Conformity assessment bodies are free to include information above and beyond that called for by the guidance document. When doing so, they should endeavour to categorize the additional information according to the four parts of a report identified in GD211 and to insert it in the appropriate section of the audit report. Additional information should not detract from, nor contradict, the required content.
The guidance document presents a preferred order and grouping of information. It is recognized that some elements may be relocated within the report due to operational requirements of CAB's; however, this practice should be minimized.
As a first step in applying the guidance in GD211, auditors should critically review their own reporting practices and build on their strengths. It is understood that applying the new guidance will require both time and practice, and that auditors are unlikely to excel in their first attempts. A disciplined approach to self-review and critique will eventually lead to proficiency.
While a report prepared according to the guidance in GD211 may be longer and more detailed than what is typical for many CAB's, this should not have a direct affect on actual auditing practices. Although reports may now contain more detailed information, report authors should resist the temptation to write lengthy reports using excessive detail and language. Short, descriptive, and factual sentences should be used to convey the necessary information.
The Medical Devices Regulations and their implementation set out a number of situations where a manufacturer submits a valid certificate to the Medical Devices Bureau. Such certificates, issued by Health Canada recognised registrars, are an attestation on the part of the issuing registrar that the quality management system of the manufacturer in question has been audited against ISO 13485:2003 in accordance with Health Canada's requirements, and has been found to be in conformity with this standard for the scope of activities as outlined on the certificate. All audits must be supported by a written report, the content of which must meet requirements found in ISO/IEC 17021:2006, ISO 19011:2002, and Health Canada's guidance document GD210.
Whereas a certificate is an attestation of conformity of a quality management system to specified requirements, the corresponding audit reports represent a significant portion of the objective evidence of the implementation of the conformity assessment procedure underlying this attestation. Furthermore, the audit report serves as a written record of the audit team's determination with respect to the extent to which specified requirements are fulfilled. Therefore, the audit report serves as a basis not only for demonstrating the conformity of the quality management system, but also for demonstrating the conformity of the conformity assessment procedure itself.
To ensure a consistent and uniform application of requirements - a desirable situation in the context of a regulatory programme - criteria must be documented and sufficiently detailed to minimize subjective interpretation. The existing requirements, as documented in ISO/IEC 17021:2006, ISO 19011:2002 and Health Canada guidance document GD210 are limited to generic requirements for the content of audit reports and are subject to interpretation in their implementation.
Because of these discrepancies in the documentation of comprehensive reporting requirements, significant variations have been observed in the reporting practices of Health Canada recognised registrars. Variations have been observed with respect to the format of reports, the content of reports, as well as the depth of reporting. Such variations undermine the even-handedness of the assessment of the application of the Medical Devices Regulations. The outcome of the present situation presents a case that manufacturers are not audited to the same level of scrutiny and that the registrars do not operate in a substantially equivalent manner with respect to specified requirements. Hence, guidance specifying the content and format of an audit report prepared in support of a certification used to obtain, or to maintain, a medical device licence is necessary to ensure the uniform application of regulatory requirements.
Given that minimising variations in audit reporting practices is a desirable objective, and given that a comprehensive set of requirements, with clear interpretations, would allow Health Canada to inform registrars of its expectations and also serve as an evaluation tool, it was deemed appropriate to develop such a guidance document.
The guidance document GD211: Guidance on the content of quality management system audit reports is largely based on the technical content of Global Harmonization Task Force (GHTF) document SG4/N33R16:2007 Guidelines for Regulatory Auditing of Quality Management Systems of Medical Device Manufacturers - Part 3: Regulatory Audit Reports. The main objective in adapting the GHTF guidance was to develop an adoption that would be relevant to the Canadian regulatory context. For instance, post-market activities, which are the responsibility of the Health Products and Food Branch Inspectorate, were removed from the guidance; additional items were added such as the Company ID number field; and definitions were modified to match those used in the Medical Devices Regulations and Health Canada guidance documents.
2.2 Regulatory Cooperation
Although primarily developed as a Health Canada specific guidance document, GD211 now finds application outside the CMDCAS programme. The Pilot Multi-Purpose Audit Program (PMAP), a joint endeavour between Health Canada and the FDA, revealed that a single audit report was necessary in order for a single audit programme to be successful. Health Canada and the FDA therefore cooperated to adapt the GD211 document to not only meet the specific needs of Health Canada, but to also be acceptable to the FDA.
As a result, GD211 is now the format in which audit reports must be submitted to CDRH as part of the FDA ISO 13485 Audit Report Submission Pilot Program. Furthermore, this new pathway to regulatory cooperation and harmonization is seen as a building block of a future joint audit program initially involving Canada and the United States, and ultimately other jurisdictions as well.
A guidance document cannot address all possible situations. However, the application of principles can assist auditors in preparing audit reports in unusual circumstances while still adhering to the requirements of GD211. Additionally, auditors can also use such principles as a guide to determine if audit reports contain sufficient audit evidence and adequately substantiated findings and conclusions.
In this context, the following four principles should be kept in mind when preparing audit reports:
- Fair presentation;
- Evidence-based decisions making;
- Responsibility; and,
- Positive reporting.
3.1 Fair presentation
The principle of fair presentation, taken from ISO 19011:2002, is described as "the obligation to report truthfully and accurately". This means that audit findings, audit conclusions and audit reports reflect truthfully and accurately the audit activities. Significant obstacles encountered during the audit and unresolved diverging opinions between the audit team and the auditee are reported.
Auditors should always include significant obstacles encountered in their audit reports. The knowledge of these obstacles will assist readers in understanding the context in which audit evidence was examined and audit conclusions were drawn. Obstacles can affect the reliability of audit conclusions and users of audit reports are best served by understanding this.
As is common, and normal in audit situations, auditees sometimes disagree with the audit team's findings and conclusions. Occasionally, these diverging opinions cannot be resolved through discussions during the audit and have the potential to distract the audit team and detract from effective auditing. Rather than change or downplay contentious findings and conclusions, audit teams should simply report their findings and indicate that unresolved diverging opinions remain. By ensuring that the report contains sufficient audit evidence and context for such findings, auditors can rest assured that their position will be presented to the certification body if the auditee appeals the finding.
Applying the principle of fair presentation also means that auditors should not shy from reporting situations where auditees are unable to provide adequate evidence of conformity. Auditors should also feel free to report on the maturity of the QMS and the auditee's quality culture. Users of reports benefit from this type of information as it provides additional context for understanding the audit team's findings and conclusions.
3.2 Evidence-based decision making
ISO 19011 describes the evidence-based approach to decisions making as "the rational method for reaching reliable and reproducible audit conclusions in a systematic audit process." A key aspect of this approach is that audit evidence must be verifiable. Because audit evidence is based on a sample of the available information, the appropriate use of sampling is closely related to the confidence that can be placed in the audit conclusions.
In order for objective evidence to be used as audit evidence, it must be verifiable. This means that documents, records, parts, components, and finished devices must be identified. Revisions numbers, identifiers and serial or lot numbers should be recorded where necessary to allow confirmation of audit evidence. Individuals interviewed should be named or otherwise referenced: it is not possible to confirm an interview or follow-up on a discussion with an unnamed employee. While some may worry that including names of interviewees could lead to retaliation towards employees, it should be kept in mind that auditors never operate without the presence of an audit guide; such guides are typically managers and take notes of the audit evidence including the names of persons interviewed.
Where auditors are approached by an employee wishing to remain anonymous [for example (e.g., confidential informant], independent confirmation of the information provided should be obtained though alternate audit evidence.
Due to the sampling nature of audits, the audit evidence used to substantiate findings and conclusions represents only a portion of the available evidence. It is therefore important that auditors strive to use meaningful samples. Although it is not always possible, nor desirable, to select a statistically significant sample, auditors should strive to maximize the value of samples and to disclose the perceived validity of samples in terms of representing the population being sampled.
The principle of evidence-based decision-making can also guide auditors report their audit findings. When preparing audit summaries, auditors can gauge if the level of audit evidence contained in the summary is adequate by determining if it is sufficient to make an evidence-based decision in accordance with their findings and conclusions given the criteria in question. Hence, auditors can tell if the summary contains 'enough information' if the evidence presented therein points to the findings and conclusions.
ISO/IEC 17021, when discussing the principle of responsibility, states:
"The certification body has the responsibility to assess sufficient objective evidence upon which to base a certification decision. Based on audit conclusions, it makes a decision to grant certification if there is sufficient evidence of conformity, or not to grant certification if there is not sufficient evidence of conformity."[ISO/IEC 17021:2006 4.4.2]
The principle of responsibility is significant in that it informs auditors that they are to assess sufficient objective evidence of conformity to substantiate any findings and conclusions. More importantly, responsibility makes it impossible to make a decision to certify where there is insufficient evidence of conformity. This leads to the concept that an absence of evidence of nonconformity is not evidence of conformity in itself.
3.4 Positive Reporting
The final principle intended to guide report authors is that of positive reporting. This principle brings together the preceding three: in order to support responsible evidence-based certification decision-making by the certification body, auditors assess and report sufficient audit evidence to support their findings and conclusions. Auditors supplement their reports by including obstacles to the audit and diverging opinions between themselves and the auditee [that is (i.e.) fair presentation] in order to provide context.
The implication of positive reporting is that all certification decisions must be based on audit findings and conclusions that are substantiated by audit evidence in the audit report. Therefore, and area not addressed in the report is assumed to be an area not audited. To assume otherwise implies that certification decisions do not need to be supported by objective evidence in contradiction with the principles of evidence-based decision making and responsible certification.
4.0 Users of Reports
In a regulatory context, like a medical device quality management system audit programme, the regulator is the ultimate user of certifications and audit reports.
Regulators use certifications and audit reports to grant market access to manufacturers and devices. They also use audit reports to recognise the work of third party certification bodies and as part of preliminary investigations into post-market issues. Regulators therefore require that audit reports contain sufficient information to identify the manufacturer, describe the audit parameters, support findings, and to conclude on the overall conformity and compliance of the manufacturer.
5.0 Report Format
Audit reports should be typed. They should be in a common electronic format that can easily be stored and transferred. It is advantageous to have reports in a format that allows keyword searches although this is not essential.
Reports should constitute a single document whenever possible. Recognizing that this is not always possible, efforts should be made to minimize the proliferation of documents that constitute an 'audit report'. References to other documents as primary sources of information should be avoided in most cases except where the volume of information is prohibitively large. This means that auditors should refrain from constantly referring the reader to additional documents to obtain the required information. This does not however preclude the use of supporting documents as appendices to the report. When appendices are used, they should be identified and referenced in the report. Appendices form part of the report and should be stored with it.
Keeping in mind the preceding paragraph, registrars and certification bodies are free to use reporting formats that meet their operational needs. However, the audit reports they produce should contain all the mandatory information from section 2.3 of GD211. This information should be arranged along the broad categories identified in the guidance document; namely information about the manufacturer, information about the audit, audit findings, and conclusions. The ordering and grouping of information presented in GD211 is strongly preferred by regulators.
Finally, audit reports can contain information above and beyond that called by GD211. When auditors do opt to include additional information, they should strive to categorize it according to the four parts identified in GD211 and to insert these additional details in the appropriate section of the audit report.
6.0 Report Language
The language of audit reports is subject to the operational needs of the certification body but should nonetheless be understandable by the manufacturer. Typically, the reporting language is subject to agreement between the certification body and the manufacturer prior to the audit. Nevertheless, audit reports intended to be submitted to regulators should be in a language determined by the regulator. As a matter of policy, reports generated as part of the Canadian Medical Devices Conformity Assessment System (CMDCAS) should be in French or English, or be made available in either language upon request by the regulator. Similarly, reports submitted to the United States FDA's Centre for Devices and Radiological Health (CDRH) must be in English.
7.0 Parts of a Report
As previously mentioned, GD211, in section 2.3, organises information in four parts. These four parts are:
7.1 Information about the manufacturer
This section focuses on providing information that identifies the manufacturer and its devices. Details provided allow the reader to understand the activities carried out by the manufacturer, its general organisational situation, the key aspects of its QMS, its relationship with related facilities covered by the QMS, and its use of critical suppliers of finished devices or processes like sterilizers.
7.2 Information about the audit
In this section, details about the parameters of the audit are given. Specifics such as the audit scope, objectives and criteria are augmented by details about the audit team and the date and duration of audit activities.
7.3 Audit Findings
This section of the report comprises the details of the audit findings. Details related to major changes, obstacles, and nonconformities are also included.
This final part of the report contains the audit team's overall impressions with the QMS being audited. This section focuses on holistic determinations of conformity and effectiveness. Matters related to reliability are also addressed along with the audit team's recommendations to the certification body.
8.0 Information about the manufacturer
Audit reports should contain information that unambiguously identifies the manufacturer audited. Although this may appear to be simple on the surface, the identity of a manufacturer can take several forms depending on the perspective employed. For example, a manufacturer may have a legal and corporate identity which differs from its trade identity in terms of the marks under which it sells its devices. The manufacturer can also be part of a larger company or group of companies. A manufacturer can also be described in terms of the devices it designs, manufactures, or distributes.
The identity of the manufacturer is significant in a regulatory context due to the liability issues associated with the sale of medical devices. Regulators hold certain entities responsible for marketed devices although the actual design, manufacturing, and distribution of medical devices can involve a number of related entities. Therefore, licensure of manufacturers and devices is not necessarily commensurate with the physical and organisational reality. The link between the regulatory and the physical aspects of a manufacturer is the quality management system (QMS).
The importance of information describing the QMS in the report is two-fold: it informs the reader on the link between the physical, organisational, and regulatory aspects of the manufacturer, as mentioned above, and it also identifies the primary object of the actual assessment. It is therefore key that the report adequately describe the QMS audited. This description should address the scope of the QMS in terms of the activities performed by the manufacturer, the activities that are outsourced and managed through the QMS, and the products that are designed and/or manufacturer under the QMS.
8.2 Specific Requirements
GD211, in section 2.3.1, identifies the following items as content requirements in relation to information about the manufacturer:
a) Manufacturer's Name and Address
The name and address of the manufacturer subject to the conformity assessment procedure, as it will appear on the registration certificate, should be included in the report.
This should include the address of all locations/facilities covered by the registration and included on the certificate of registration pertaining to the audit. If more than one address is included, than the main or primary address on the certificate and the main or primary address in the audit report should match.
The address included in the audit report should not be limited to a postal address if this differs from the physical (or municipal) address of the location(s) audited; the point of interest is not how to have mail delivered to the manufacturer, but how to locate its facilities.
b) Company Identification Number
The manufacturer's 'Company ID' number assigned by Health Canada should be obtained from the MDALL website and included in the audit report in association with the manufacturer's name and address. Where a company has no licensed devices, no Company ID number will exist. In such a case, a notation of 'N/A' or 'not applicable' should be made.
Other regulatory identifiers (e.g. United States FDA Federal Establishment Identifier (FEI) number) can also be included. Care should be taken to indicate to which regulatory scheme each identifier belongs.
c) Corporate Identity of the Manufacturer
When a manufacturer has multiple names or identities these should be clarified. This clarification also extends to any relationships with sister, parent, and daughter companies, including subsidiaries, acquisitions, business units, and joint ventures. When preparing this section, auditors should be mindful to frame the explanation in the context of the QMS being audited and its associated scope of activities and devices.
This item can be omitted from surveillance audit reports.
As mentioned above, the identity of a manufacturer can take many forms depending on the perspective applied. This item seeks to address matters of divergence between the entity authorised by regulators and the market identity of the manufacturer. It should be presented in the context of the QMS that is subject to audit.
This section is not intended to be an in-depth analysis of the corporate holdings of the manufacturer. It should however answer the following questions:
- Under what names does the manufacturer present itself to the marketplace?
- Under what names does the manufacturer market its devices?
- Is there any regional/geographical delineation to the names used?
- Does the manufacturer market itself as part of a broader corporate group?
- Does the manufacturer market itself under a name or mark that it does not own?
- What is the involvement of related companies (parent, sister, daughter, etc.) in the design, manufacturing, and distribution of medical devices controlled by the QMS under audit?
The description of the corporate identity of the manufacturer can be omitted from surveillance audit reports if the following two conditions are met:
- The corporate identity of the manufacturer has been previously described in a certification or re-certification report; and
- There have not been any changes in the information since it was last reported.
Where this item is not included in a surveillance audit report, a mention that the information previously reported remains accurate and unchanged should be included.
A. <Company> operates as <Company> in Canada but also markets some of its products under <Other Brand> in the US.
B. <Company> operates as <Brand Name>. Devices are sold under the <Brand 1> banner in North America and Japan and <Brand 2> in Europe and Latin America.
C. <Company> is a wholly owned subsidiary of <Big Group> and labels its product as <Company>, a <Big Group> company. Devices are sold under the generic <Big Group Brand> trademark owned by <Big Group>. <Company> uses marketing and distribution channels of <Big Group> for all of its products.
D. <Company> is part of <Big Group> group of companies. Devices are marketed under the <Company> brand. Design is outsourced to <Sister Company> design centre. Devices are distributed and warehoused by <Big Group> corporate distribution.
E. There have been no changes to <Company> corporate identity since the last certification audit.
d) Description of the Manufacturer
A description of the manufacturer should be included in the report. This description should include the approximate number of employees and associated number of shifts. The description should also include an overview of the activities and processes carried out by the manufacturer at the audited location(s) as well as identification of key outsourced activities. The name and title of senior management of the location(s) audited should be included in the description.
Where the conformity assessment procedure involves more than one physical site, all sites should be identified [as in a) above] and a description of the relationships between the sites and their relative role within the QMS, including any shared functions, should be included.
The description of the manufacturer can be limited to those parts that fall within the scope of the audit for surveillance audit reports.
The audit report should provide a clear and accurate description of the manufacturer and its activities. The information in the description of the manufacturer, in conjunction with the parameters of the audit reported elsewhere in the report, provides context for the audit findings and conclusions. This also serves to validate the appropriateness of the audit scope given the risks associated with the manufacturer's activities. Without an adequate description of the manufacturer, it is not possible to determine the suitability of audit coverage or to gauge the reliability of the certification for regulatory purposes.
The description of the manufacturer should address the following:
The total number of employees should be reported. This should, where applicable, differentiate between full-time and part-time employees. The report should also mention temporary employees and employees working off-site.
The description of the manufacturer should indicate the number of shifts, even if there is only a single shift. The report should detail the time periods of the shifts (start and end times) and the number of employees assigned to each shift if more than one shift exists. In cases where certain shifts only perform a limited subset of activities, it may be appropriate to mention this in the report.
The report should include an overview of the activities and processes undertaken in the manufacturer's facilities. This should address major functional areas (e.g. design) as well as major manufacturing / production activities (e.g. coating, moulding, assembly, fermentation, packaging, etc.).
Key outsourced activities should be mentioned in the overview of activities and processes. Examples of this include:
- Printing and population of PCB boards;
- Development of firmware;
- Specialised coating processes.
In very complex manufacturing situations where several facilities and outsourced steps interact, auditors can opt to append a diagram to help describe the activities of the manufacturer.
The description of the activities performed on-site and those that are outsourced is a key element of the information provided about the manufacturer. Auditors should take care to include an appropriate level of detail.
The name and title of the most responsible individual of the location audited should be included in the description. It is not necessary to list all company officers or senior managers.
If the certification includes more than one physical site, all of the locations should be described as above. This should be supplemented by a brief description of the role of each site in the QMS (e.g. design centre, manufacturing facility, head office - management only, etc.). Common and shared QMS functions (e.g. document control, CA and PA) should also be mentioned when the inter-relation of QMS sites is discussed.
Changes to the manufacturer can also be highlighted in this section of the report.
For surveillance audit reports, the description of the manufacturer can be pared down to the sites included in the audit and the manufacturing lines covered.
A. <Company> is a small privately-held company employing 42 people on a single shift at its <City> location. The company designs and manufactures acrylic teeth for restoration. All activities are performed in-house in its 25,000 square foot facility. Key activities involve production of polymer powder, mould design and machining, injection moulding, and setting. The most senior manager at the site is <Name>, the Chief Executive Officer and owner of the company.
B. <Company> operates two shifts at its 40,000 square foot facility. The first shift (0700-1500) has 96 employees and the second shift (0900-1700) has 44. The company also employs 6 field service personnel and 3 field sales reps. (total 149 employees.) The company designs and manufactures gas flow-meters and distributes related accessories purchased from other manufacturers. Key activities at the facility are machining and inspection of parts, assembly, and calibration of gas-flow meters.
The company also operates a 32,000 square foot warehouse and distribution center in the same industrial park. The warehouse has 15 employees on a single shift (0900-1700). <Name> is the company's general manager and is the most senior manager for both sites.
C. <Company> employs 68 people on a single shift at its dental implant manufacturing facility. Key activities include design, machining, and shaping. Passivation and coating is performed by a supplier, as is sterilisation. Distribution and marketing is performed by <Sister Company>, a sister company on behalf of the parent group <Big group>. <Sister Company> operates an independent QMS. Both <Company> and <Sister Company> must follow generic QMS policies of <Big Group>. <Name> is the General Manager of <Company> and reports directly to the board of <Big Group> as the most responsible individual at <Company>.
e) Scope of Certification
The report should include the scope of certification of the manufacturer being audited. This includes activities and a list of the generic medical device groups or families that are included in the scope of certification. Where the scope of certification is prohibitively long, it may be referred to in an appendix.
Where the scope of certification changes as a result of the audit (e.g. during and expansion to scope audit or upgrade audit), the report should clearly identify this and allow the reader to determine the scope registration prior to the audit and what the proposed scope is following the audit.
In the case of a certification involving multiple locations, the report should provide the overall scope of certification as well as site-specific sub-scopes.
For guidance on the formulation of scopes of certification, consult Health Canada's guidance document GD207: Guidance on the Content of ISO 13485 Quality Management System Certificates Issued by Health Canada Recognized Registrars.
For large and complex scopes of certification, it may be appropriate to append the certificate to the report and to refer to the reader to appendix.
f) Identification of Critical Suppliers
The report should identify the name, address, and product or service of critical suppliers that provide products or services used in the audited processes. The involvement of a supplier can be through an outsourced process such as sterilisation or software development. Where the list is prohibitively long, the report may refer to an appendix.
This item can be integrated in the Audit Findings section of the report
GD211 borrows the definition of critical supplier from the GHTF document SG4/N84:2010 Guidelines for Regulatory Auditing of Quality Management Systems of Medical Device Manufacturers - Part 5: Audits of Manufacturer Control of Suppliers. The term critical supplier is defined in GD211 as:
[A] supplier delivering materials, components, or services that may influence the safety and performance of the product.
Note: In the context of the audit of medical device manufacturers, a critical supplier is a supplier of a product or service, the failure of which to meet specified requirements could cause unreasonable risk to the patient, clinician or others, or could cause a significant degradation in performance. This can include suppliers of services which are needed for compliance with QMS or regulatory requirements.
The definition above can lead to some ambiguity in certain cases. It is important to realise that the identification of a supplier as 'critical' is the product of a thoughtful and methodical approach to risk management on the part of the manufacturer. In general, auditors should rely on the manufacturer's determination of a supplier's criticality. However, in cases where an obviously critical supplier is not treated as such, auditors can pursue the matter by asking for evidence of risk management, purchasing specifications, and associated control measures applied to supplied parts and services; nonconformities should be issued where appropriate.
The identification of critical suppliers in the audit report serves a number of purposes. It affords the reader a better understanding of the scope of activities of the manufacturer and of how its devices are manufactured. It highlights the relative importance of supplier control and incoming inspection activities. Finally, it allows an assessment of the suitability of the audit by highlighting possible areas of risk not addressed by the audit.
The audit report need only contain information on critical suppliers that are involved in the audited activities. The critical supplier of a part or service used in an area that is not part of the scope of the audit does not need to be included in the audit unless it forms part of the sample reviewed when auditing purchasing controls and related activities.
Report authors can opt to include the information related to critical suppliers in the audit summaries or under a separate heading in this part of the report (information about the manufacturer.)
g) Contact Person for the QMS
The name and contact information of the contact person for the QMS should be included in the report.
The report should include the name and contact information for the most appropriate person to contact in relation to the QMS. This person should be identified by the manufacturer, as it is not necessarily the Management Representative or the Quality Assurance (QA) Manager. The contact information provided should include a phone number or email address.
h) Status of any Relevant QMS Certification
If not apparent elsewhere in the audit report, the status of any relevant certification or registration of the QMS of the manufacturer should be listed.
The reality of today's globalized medical device industry is that manufacturers sell their devices in numerous jurisdictions. As a result, many manufacturers hold multiple certifications of their QMS to ISO 13485:2003 issued under various regulatory schemes. The existence of such certifications, as well as their status, speaks to the stability, suitability, and maturity of the manufacturer's QMS and corporate quality culture. It also gives an indication of any additional oversight that the QMS may be subject to. Therefore, including information about additional QMS certifications in the audit report provides users of reports with information allowing them to better form a judgement regarding the reliability of the audit and of the QMS.
In this context, relevant certifications are those related to a medical device regulatory scheme (e.g. CE marking under the In-Vitro Diagnostic Medical Device Directive (IVDMDD), the Medical Device Directive (MDD), or the Active Implantable Medical Device Directive (AIMDD), Japan Pharmaceutical Affairs Law (PAL), etc.) A certification does not need to be the subject of the audit of the report to be relevant. To be considered relevant, a certification should cover the same manufacturer, facilities, and medical devices (or very similar devices.)
The status of a relevant certification indicates whether the certification is in good standing, is suspended, or has been withdrawn or cancelled.
i) Exclusions and Non-Applications of Requirements in the QMS
Where the manufacturer being audited has claimed an exclusion or non-application of requirements of ISO13485:2003 in its QMS, these should be identified in the report. The report need not include the justification of these exclusions and non-applications.
Audit reports should identify any exclusion and any requirements that are not applicable. The 'Application' section of ISO 13485:2003 allows for the exclusion of design and development controls where permitted by regulations. The standard does not allow for any further voluntary exclusion to its application.
Because ISO 13485:2003 is such a broad standard that is designed to apply to all types of medical devices, it is inevitable that some of the requirements therein will not be applicable for certain manufacturers. When a manufacturer does not apply a given requirement in the standard, a justification for not applying the requirement should be recorded in the QMS documentation.
While ISO 13485:2003 indicates that non-applicable requirements are limited to section 7 of the standard, it is understood that item 18.104.22.168 Particular requirement for active implantable medical devices and implantable medical devices will only be applicable to those manufacturers making implantable and active implantable devices.
Auditors should list all exclusions and non-applications in every report, as these are a modifier to the scope of certification and assure the reader that no part of the QMS was overlooked. While the manufacturer is required to document the justification for these exclusions and non-applications, auditors are not obliged to include these justifications in the audit report.
9.0 Information about the audit
The audit report should describe in adequate detail the nature and parameters of the audit performed. As part of an evidence-based decision making process, the audit is a sampling exercise; report authors should therefore explain the overall sampling methodology represented by the audit parameters.
An understanding of the audit parameter is necessary in order for the reader to understand the context of the audit, and in particular the extent of coverage of the audit. Describing the audit parameters in part demonstrates that accreditation, recognition, and certification rules were followed. Identifying audit team members provides the opportunity to confirm a number of matters including competence and impartiality, as well as provides names for future follow-up activities and clarifications should it be necessary.
Keeping in line with the principle of fair presentation, describing the specifics of the audit performed can also disclose factors that have the potential to affect the reliability of the audit such as the use of multiple languages, large audit teams, complex scopes, and the use of interpreters and translators.
9.2 Specific requirements
The following items are identified as content requirements in section 2.3.2 of GD211:
a) Audit Type
The report should identify the type of audit performed (eg. certification, surveillance, re-certification, etc.)
When describing the audit type, auditors should use clear and meaningful words. The reader should understand whether the audit performed was a full audit covering the entire QMS, or a partial audit covering only part of the QMS.
If the audit includes several criteria or regulatory schemes, and the type of audit varies from one criterion to the next, the audit report should clarify this (e.g. a re-certification audit for ISO 13485:2003 under CMDCAS and a surveillance audit for ISO 9001:2000).
b) Audit Criteria
The audit criteria should be listed in the report. For audits performed under the CMDCAS programme, this would normally include, as a minimum, ISO 13485:2003, the applicable regulatory requirements as stated in the Regulations, and the manufacturer's QMS documentation.
The audit report should include the audit criteria. The identification of criteria should be unambiguous, in particular where national adoptions of the criteria documents exist (e.g. CAN/CSA ISO 13485:2003). Auditors should refrain from using non-specific terms such as 'applicable regulatory requirements' or 'Canadian regulations' to identify audit criteria; opting instead to use the appropriate title of criteria documents (e.g. Part 1, Canadian Medical Device Regulations or Title 21 US Code of Federal Regulations Part 820, etc.)
c) Audit Objectives
The audit objectives should be listed in the report. This includes, as a minimum, the following:
- the assessment of the conformity of the manufacturer's QMS to ISO 13485:2003; and
- the assessment of the capability of the QMS to ensure compliance with applicable regulatory requirements. The applicable regulatory requirements should be clearly identified in the objectives.
Audits may also have additional objectives such as the evaluation of the effectiveness of the management system in meeting its specified objectives or the follow-up of nonconformities issued during previous audits.
If the objectives of the audit vary depending on the audit criteria (and possibly the type of audit associated with each criterion), the report should clarify this so that no ambiguity exists in relation to the audit objectives.
d) Audit Scope
The report should include the scope of the audit. Particular attention should be placed on the physical locations and organizational units of the audit and, in the case of a surveillance audit, on the activities and processes that form the scope of the audit.
According to ISO 19011:2002,
"The audit scope describes the extent and boundaries of the audit, such as physical locations, organizational units, activities and processes to be audited, as well as the time period covered by the audit."
This implies that the statement of the scope of the audit should be informative and meaningful in terms of identifying the physical and organisational locations to be audited including the actual activities or processes subject to audit. For the purposes of GD211, the audit scope does not need to include the time period covered by the audit.
Note: The scope of the audit should not be confused with the scope of certification. A well written scope of an audit will always differ from the associated scope of certification, even for a certification audit.
A. This surveillance audit is limited to the management activities (resource management, management review, planning), the production line of the <Device Name> CK-MB rapid assay, the incoming inspection, and QM activities (internal audit, Material Review Board, CAPA, post-market surveillance incl. complaint handling) located in the main building at <Address 1> and its annex <Address 2>.
B. As this is a re-certification audit, all QMS processes are included in the audit. Design and management functions at <Site 1> will be audited as will the production of vascular access ports, PICC catheters, and haemostasis devices at <Site 2>. Support functions (QA, Shipping/Receiving, Facilities) will be audited at their respective locations.
C. The scope of this audit is focused on the mandatory management processes, polymer powder production, infrastructure, calibration, and customer related processes. All activities take place in the <Site Address> facility.
D. This special audit will focus on design controls, production, traceability and post-market surveillance activities related to the <Model X> Automatic External Defibrillator (AED) and in particular the software associated with the device.
e) Audit Dates
The dates of the on-site audit should be included in the audit report. This should also include the number of auditor-days on-site.
Report authors should be mindful to use a dating convention that is unambiguous or to indicate the dating convention in the report (e.g. yyyy/mm/dd). If certain audit team members are only on site for certain parts of the audit, then the attendance of the various team members should be clarified.
f) Identification of the Audit Team
The report should identify all members of the audit team and describe their respective role (e.g. team leader, technical expert, etc.). Any observers present should also be listed. Where interpreters are used, they should be identified. The affiliation of interpreters should also be indicated.
Observers are people attending the audit that are neither associated with the manufacturer nor auditors or technical experts. Examples of observers include accreditation body assessors or regulators performing witness audits or registrar staff conducting performance evaluations or witness audits. Because the presence of observers is known to affect the performance of auditors, it is important to disclose their presence in the report.
When interpreters are used, this should be mentioned. The affiliation of interpreters should be indicated in the report. Interpreters could be contracted by the registrar or the manufacturer. Interpreters may also, in some cases, be employees of the manufacturer. Because interpreters in effect filter objective evidence, it is important to disclose their use and affiliation.
g) Audit Language
The language or languages used during the audit should be indicated in the report.
Auditors should also mention languages used informally to interview staff if these differ from the official languages of the audit. If such use of other languages during the audit occurs and interpretation is required, the identity of the interpreter should be disclosed in the report. Interpreters used during an audit are not always professional interpreters; sometimes another employee or supervisor is used as an interpreter. In such cases, the identity of the interpreter and his/her affiliation should nonetheless be recorded.
h) Document Review Results
When a review of the manufacturer's QMS documentation is performed prior to the audit, this should be mentioned in the audit report and reference to both the report and the results of the review should be made.
10.0 Audit findings
The audit report should include sufficient audit findings, both positive and negative, to support the audit conclusions made in the report. Audit findings should always be framed in context through objective evidence and evaluated against the appropriate audit criteria.
Because the audit report is a record of what was reviewed and the audit team's conclusions, omission of an aspect of the audit of the manufacturer's QMS in the report is taken as an area not audited. The absence of detected nonconformity does not automatically imply conformity; evidence of conformity must be presented in the audit report in order to support a conclusion of conformity.
Reports should not contain opportunities for improvement, including specific advice, instructions, or solutions towards the development and implementation of a QMS. However, as an important component of a complete and accurate record of the audit, observations and findings should be reported. Observations can include situations which appear to be non-conforming but where insufficient audit evidence was collected. Where there is an observation that is not supporting conformity (i.e. a negative finding) but that is not a nonconformity, it should be stated in a factual and neutral manner. A potential solution should not be suggested. Words such as "consider" should be avoided.
In this context, the term 'observation' should not be confused with 'Observations' that would be reported on a FDA Form 483 (these would constitute nonconformities). The observations in question are observations of questionable significance that would nonetheless be discussed with management and included in the Establishment Inspection Report (EIR).
A. There is no direct link between the OEM lot number and the lot number assigned by receiving in the Device History Record. A lookup table of receiving records must be used to trace parts back to the OEM lot number.
B. IEC 60601-1 is not applicable to software-only devices.
C. Hard-copy records that have been entered into the electronic system are not identified as having been entered.
D. The distribution agreements do not stipulate the frequency at which distributors must send copies of distribution records to the manufacturer. The agreements only mention that it must be done.
E. The Standard Operating Procedure (SOP) <XX-XXX> requires that operators cut the catheter to a length between 14.961 and 15.157 inches. The design specification is for a length of 15.1 ± 0.1 inches. The measurement acuity of the rulers provided is 0.05 inches.
F. Calibration records do not include ambient conditions in the metrology lab. These can be obtained from the timestamp on the calibration records and the ambient conditions log of the metrology lab.
G. Employees can only access their job description through the human resources office.
H. Managers must sign training records to indicate that the training was deemed to have been effective. The method by which effectiveness was determined is not recorded.
10.2 Specific Requirements
Section 2.3.3 of GD211 contains the following requirements:
a) Audit Summaries
Written summaries of the audit of each QMS process or activity audited should be included in the report. Examples of QMS processes or activities include:
- management processes (management review, resource management, internal audits, organizational structure, training, etc.);
- design and development;
- production and process controls;
- corrective and preventive action systems;
- purchasing controls;
- control of documents and records; and
- customer related processes.
Note: the above list is not meant to be all inclusive and is included for illustrative purposes only.
The audit summaries should be brief but nonetheless include the following information:
- description of the QMS process or activity audited;
- area (physical or organizational) of the site visited;
- name and title of persons interviewed;
- key documents reviewed (procedures, work instructions, etc.);
- type and number of records reviewed, including a qualitative statement of the sample size where appropriate;
- identification of products or components reviewed; and
- statements regarding the conformity of the activity or process under audit to the audit criteria.
Note: the inclusion of clause numbers in the concluding statements can help demonstrate appropriate coverage.
Written summaries of the audit of each QMS process or activity audited should be included in the audit report. Summaries should be arranged by audit topic or QMS process. There are many ways to do this, such as:
- by QMS process as identified by the manufacturer;
- by section of the standard;
- by subsystem.
It may also be practical to organise the audit summaries in accordance with the audit plan.
The key to writing summaries that are both brief and meaningful is to ensure that they include appropriate amounts of context, evidence, criteria, and evaluation. The sum of these four elements should be a finding, either of conformity or non-conformity:
The summaries should include context in the form of the description of the QMS process or activity audited as well as the area of the site visited. The context provided should be sufficient to allow the reader of the report to understand the evidence presented in the summary.
The summaries should include audit evidence. Audit evidence is defined as objective evidence that is verifiable. It is therefore necessary to identify in the report the persons interviewed during the audit. The report should also identify the documents reviewed by the audit team. Identifying information should include document and revision numbers where appropriate. The type and number of records reviewed should also be described in the report. Auditors should strive to qualify their sample of records either numerically (e.g. 15 out of 67 records were reviewed) or qualitatively (e.g. a small sample of 12 records was reviewed). The report should also identify any products or components reviewed during the audit - this can be done by including, for example, part numbers, model and serial numbers, or batch or lot numbers.
Although there are no easy rules by which a report author can gauge whether enough audit evidence has been included in the audit report, there are three guiding concepts to apply.
The first concept to consider, which was presented above, is the principle of evidence-based decision making. The report authors can, by contrasting the audit findings with the provided audit evidence, determine if sufficient evidence is presented to substantiate the findings and ultimately the conclusions of the audit without the need for assumptions. The question to answer is whether the evidence presented in the report is sufficient on its own to allow the reader to reach a conclusion similar to that of the report author.
The second concept is the significance and risk of the audited activity or process. Activities and processes that have a high significance in terms of the scope of registration (e.g. design, manufacturing) or the safety and effectiveness of the medical devices, or that are complex chains of activities associated with a number of requirements from the criteria, will necessarily require more descriptive detail and audit evidence in the audit summaries.
The third guiding concept is the presence of findings of non-conformity. When an audit reveals findings of non-conformity, the associated audit summaries should provide sufficient details describing the process or activity and objective evidence to ensure that the manufacturer has the right information to take appropriate correction and corrective action, and to allow other users of the report, particularly regulators, to pass appropriate judgement on the significance of the finding from a regulatory compliance perspective. Auditors should not understand this to mean that when there is a finding of conformity that this finding does not need to be substantiated by audit evidence in the report.
Audit summaries should identify the applicable criteria unless it is clear from the description provided what criteria are applicable (e.g. internal audit). Explicitly identifying the applicable audit criteria, in particular in the concluding statements of the audit summaries, is a good way to demonstrate complete coverage. Audits of complex activities or processes will usually involve many criteria (e.g. ISO 13485:2003 6.2.2; 6.4; 7.5.1; 7.5.3; 8.2.3; etc.).
The end of a well written audit summary is a clear and concise statement of finding regarding the conformity of the activity or process audited. This need not be an elaborate conclusion; it should simply state whether the audit activities conform and, if not, identify any nonconforming aspects. Concluding statements should not contradict the objective evidence presented in the summary, nor should they represent assumptions. Report authors should feel free to qualify their concluding statements where the findings are based on small samples or weak evidence.
|Process Name||Internal Quality Audits (IQA)|
|Relevant Criteria||ISO 13485:2003 clause 8.2.2|
|Description/Findings||Internal audit objectives are set annually by the Vice President (VP) Operations <Name> based on business and quality objectives. Responsibility for developing and implementing the audit programme rests with the Quality Assurance (QA) Manager.
An audit programme was developed for 2010-2011 which included objectives related to review of progress on waste-cutting measures and implementation of new electronic records system.
The 2010 audit was performed October 6th to 9th by <Name>. <Name> works in receiving and is suitably trained (ISO 9001 auditor course (BBI), 13485:2003 course by MedForward Academy, C-MED consultants MDR training) to perform internal audits. The audit report was reviewed and met the programme objectives as well as requirements of 13485. Findings were well articulated and supported by objective evidence. The report was formally presented to the Chief Executive Officer (CEO) and VP Operations during the last Strategic Review and Planning (management review) October 27th.
All nonconformities issued during the IQA were entered into CapTrack for timely resolution (5 of 7 closed at date of this audit).
The internal audit process appears to be robust and to be well-tailored to the company's business and quality objectives.
The output of this process is judged to be reliable.
|Area Visited||QA dept., also Management|
|Persons Interviewed||<Name>, QA Manager
<Name>, VP Operations
|P-IQA-01 Rev. 2 - Internal Audit Procedure
internal audit programme 2010-2011
Internal audit report for 2010
|Product/Components Reviewed||Not applicable (N/A)|
|Conclusions||The IQA is in full conformity with the requirements of 13485.|
|Process Name||Polymer powder production|
|Relevant Criteria||ISO 13485:2003 clause 7.5.1 (excl. 22.214.171.124.2 - 126.96.36.199)|
|Description/Findings||The polymer powder used to produce <Company> products is a MMA and butyl acrylate copolymer. The polymer powder is produced by aqueous phase suspension polymerization.
Production scheduling is determined on a weekly basis through production planning meetings. Schedules are passed to the Production Manager, who reports to the VP operations.
The polymer powder is produced based on a proprietary recipe document in procedure P-PRO-01. Production Technician <Name> explained and demonstrated the major steps in the process and the equipment used during the polymerization of lot 4789-08:
The first step in production is pre-mixing of monomers according to recipe proportions. Mixing times and intensity are controlled parameters and are recorded in the batch history record (BHR) along with the lot number of the raw materials.
The main polymerization reactor is prepared concurrently. An aqueous solution is mixed according to the recipe. Amounts of buffers, salts and other additives are recorded in the BHR. PH and conductivity measurements are also taken.
The monomer solution is then suspended in the aqueous phase using an emulsifying mixer. Specific times and energy density are required by procedure and are recorded in the BHR. Samples are drawn to optically verify droplet size of the dispersion. Temperature and pressure in the reactor are increased following a specified programme during the suspension phase.
Once suitable temperature, pressure, and dispersion is achieved, a BPO initiator is added to the reactor. Polymerization rate is controlled through pressure and temperature which are critically controlled parameters. Control charts are produced and kept in the BHR. Polymerization is halted using a chain terminator once the specified reaction time has been reached.
The polymer powder is then filtered out, washed in an alkaline solution and dried. Samples of the dry powder are taken to test for bulk density, density, melt viscosity, particle size distribution and additive residuals. The results of all tests are recorded in the BHR.
Finally the powder is sifted to remove dust and large particles. The powder is then released for use by authorization of the production manager following a review of the BHR as evidenced by his signature on the lot traveller and in the BHR.
BHR's for lots 3386-07, 3399-07, 4200-08, and 4789-08 were reviewed and found to be complete and in order. All pertinent data, as per procedure, are recorded. This is deemed to be a small but reliable sample since all information was in order.
It was observed that there is no formal system in place to bridge data between the old paper BHR's and the new electronic BHR system when a lot of additive spans both systems.
The polymer powder production process is well established and controlled. However, a nonconformity was issued (see NC1-S1-08) against 188.8.131.52 since the identity of the pH meter used in the preparation of the wash solution is not recorded.
|Area Visited||Production/Polymer lab|
|Persons Interviewed||<Name>, Production Manager
<Name>, Production Technician
|P-PRO-01 Rev. 2 - Polymer powder production
Batch history records for lot numbers 3386-07, 3399-07, 4200-08, and 4789-08
Lot traveller for 4789-08
|Product/Components Reviewed||In-production batch (lot 4789-08)|
|Conclusions||The polymer production process conforms to the requirements of 7.5.1. The identified NC does not affect the ability of the QMS to ensure products are produced according to specifications.|
|Description/Findings||Calibration is the responsibility of the Quality Assurance (QA) department. A QA technician, <Name>, explained the process:
All instruments requiring calibration are identified with an asset tag number. The number on the asset tag is used to track the instrument in the database. Instruments were observed to have seals on the adjustments to prevent tampering.
A significant sample of entries (6) in the database was reviewed. All observed entries have a specified calibration interval. The database produces a weekly report of instruments in need of calibration in the following week so that they may be taken out of circulation prior to calibration expiry. Calibration interval is based on manufacturer recommendation, frequency of use, past history, and criticality of measurement.
Calibration status is also identified in colour in the database (green = calibrated, yellow = needs calibration within 3 months, red = calibration expired, blue = under calibration)
The database also indicates calibration standards and ranges for each instrument.
A number of instruments are sent for outside calibration. These are sent to approved calibrating labs found on the approved vendor list (Rev. 37 Nov 2008). All verified calibrations are traceable to national standards as per calibration certificates. Certificates of calibration are supplied with each returned instrument and filed in the QA dept. (certificates for 1324, 1398, 1222, 1557, 1752 were verified).
Some calibrations are done in-house. For example, the UV/VIS Spectrophotometer (asset tag 1473) is calibrated in-house using reference cells. This is outlined in procedure P-CAB-02 Rev. 2 and was demonstrated by <Name>. A record is generated in lieu of a calibration certificate and is filed appropriately.
All out-of-tolerance instruments have their significance assessed for product risk. However, when no action is taken, no record is generated as to the justification for not taking action; this is left as a nonconformity (NC2-S1-08).
|Area Visited||QA Dept.
|Persons Interviewed||<Name>, QA Technician|
|P-CAB-01 Rev. 7 - Calibration
P-CAB-02 Rev. 2 - UV/VIS Spectrophotometer calibration
Calibration Database (entries for 1324, 1473, 1398, 1222, 1557, 1752)
Calibration certificates (for entries above)
Approved vendor list Rev. 37 Nov 2010
|Product/Components Reviewed||N/A (not applicable)|
|Conclusions||Asides from the nonconformity (NC) issued, this process is in conformity with the requirements of 7.6. The identified NC does not materially impact the validity of measurements performed.|
|Process Name||Infrastructure and Maintenance|
|Relevant Criteria||ISO 13485:2003 clauses 6.3 and 6.4|
|Description/Findings||The infrastructure, work environment and maintenance thereof is the responsibility of <Name>, Facilities Manager. <Name> reports directly to the Vice President (VP) Operations.
<Company> owns the building in which it operates. There are no special environmental requirements in production beyond the provision of adequate space, comfortable temperature, and adequate ventilation. However, additional fireproofing and fire suppression is installed in the polymer labs and storage areas.
Facilities management is also responsible for maintaining the RO water system used in the production process. Replacement of filter cartridges is based on use (measured flow) in accordance with manufacturer recommendations. The pump and valves are also maintained on a regular basis. A separate log is maintained for the filter cartridges which records the serial number and service hours and service dates of each cartridge.
A predictive inspection and maintenance schedule is employed on critical process equipment (using CMMS software). Balance-of-plant equipment is subject to routine maintenance and replacement.
The Facilities Manager was not available during the audit due to a family medical emergency; the VP Operations answered questions and provided available records to the best of his ability. Because of this, it is recommend that this item be re-audited in depth during the next audit since findings could not be independently verified in records.
|Area Visited||Maintenance Dept.|
|Persons Interviewed||<Name>, VP Operations|
|CMMS entries for reactor agitator
RO filter cartridge replacement log
|Product/Components Reviewed||Not applicable (N/A)|
|Conclusions||A tentative conclusion of conformity is given - this should be further substantiated during the next audit.|
|Process Name||Management (including methods of monitoring the efficient operation of the quality system)|
|Relevant Criteria||ISO 13485:2003 5.X - Management Responsibility|
|Description/Findings||The company's organisational chart outlines the structure as well as functional responsibility and authorities. The Senior Director of Quality and Regulatory is identified as the management representative.
A job description (JD) for the management rep. was reviewed. The JD addresses the mandatory responsibilities from ISO 13485 as well as the responsibility for communication quality related matters throughout the organisation. <Name> indicated that he has begun sending a quarterly email newsletter to all employees to communicate quality management issues.
The executive council, composed of the Chief Executive Officer (CEO), Vice President's (VPs) and the Sr. Dir of Q and R, is responsible for all strategic and quality planning and resources. <Name>, the CEO of the company, was unavailable during the audit to discuss planning. Planning is based on business and quality objectives and takes into account process metrics (see QP-001). Planning is part of a larger quarterly meeting that also includes management review and objective setting.
The executive council conducts a management review as part of its quarterly meetings. The agenda is well defined and comprehensive. The attendees review data regarding the performance of the company, the QMS (including CA/PA), and the progress of projects. Anticipated challenges and regulatory hurdles are discussed. The council then reviews the quality objectives and policy in light of the presented data. Outputs of the meeting include updated quality objectives, policy, and operational and resource plans.
The company's quality management activities appear to be taken seriously and to enjoy adequate resources.
|Persons Interviewed||<Name>, Sr. Dir. Q and R (Mgt. Rep)|
|Planning and Objectives QP-001
Management Review QP-012
Quality Objectives QO-044-R9
Quality Policy POL-01 R3
Management Review Minutes Nov. 2010 (QP-012 R6)
Job description: Management Representative JD-011R2
Organisational Chart R22
(DRAFT) email quality newsletter for Q3
|Conclusions||Top Management has demonstrated its commitment to quality management throughout its planning and review activities. The requirements of ISO 13485:2003 section 5 are adequately addressed in the QMS. These activities are judged to be in conformity.|
|Process Name||Design and Development|
|Relevant Criteria||ISO 13485:2003 7.3 Design and Development|
|Description/Findings||Design activities are managed by the Director of Engineering.
All design control activities, including design changes to existing products, are documented in the Design Control/Design Change Manual QP-07. A "gated" process from concept to market is employed. The manual stipulates the format and content of DHF's and Technical Files.
Target markets are identified at the feasibility stage to identify all relevant regulatory requirements (for example CMDR). Risk management plans are formulated at this stage and refined as design activities progress through the various gates.
One of the outputs of the design process at <company> is the project quality plan, which brings together the DHF, the technical file, and the design transfer protocol. The project quality plan also identifies regulatory submissions at an early stage so that appropriate documentation can be collected.
Design outputs are subject to approval by all Directors to ensure that the company can implement the design.
Engineering coordinates pilot runs and prototypes with Operations. Once technical feasibility is established, design validation activities are managed by project engineers in consultation with an external medical advisor.
Although the medical advisor is not considered an employee of the company, she is not considered to be a supplier by <company>. While the Sr. Dir. of Q and R could produce a resume demonstrating her experience, her suitability has not been evaluated using the supplier control procedure as would any other supplier of goods or services. It was explained that a rigorous process had been undertaken to select a medical advisor, but that the supplier control process was not designed to handle such situations and was therefore not used. There were records of the MRB authorizing her use as an unauthorized supplier.
The final validated design is subject to a final approval by all Directors. The project engineer is then responsible for developing the design transfer protocol in cooperation with the Operations dept.
The records for the design of <Product 2> were reviewed and found to conform to the requirements of the manual and to ISO 13485. The project quality plan was comprehensive and the design transfer protocol has recently been authorised for implementation.
An additional DHF (<product 1>) was reviewed. This file included design changes. The design changes were managed through the same gated process (concept to market) as a new design.
The design control process is rigorous, systemic, and well documented. All necessary inputs are identified. The records generated by the process are well identified and organised.
|Area Visited||Engineering, Management|
|Persons Interviewed||<Name> Sr. Dir. Q and R (Mgt. Rep)
<Name> Dir. Engineering
<Name> Project Engineer - <Product 2>
|Design Control / Design Change Manual QP-07 R3
Project Quality Plan for <Product 2> R1
DHF for <Product 1> and for <Product 2>
Design Transfer Protocol for <Product 2>
|Product/Components Reviewed||Not applicable (N/A)|
|Conclusions||The design control activities at <Company> conform to the requirements of ISO 13485:2003 7.3|
|Process Name||Purchasing Controls|
|Relevant Criteria||ISO 13485:2003 7.4 Purchasing|
|Description/Findings||The specifications and acceptance criteria for purchased components and services are defined in the project quality plans for the various devices.
Supplier evaluation and incoming inspection are the responsibility of the manager of procurement who reports to the Vice President (VP) of Operations. He is responsible for the qualification of suppliers in accordance with QP-57 Supplier Approval.
Potential suppliers are evaluated based on their ability to supply products in accordance with requirements identified by the Engineering dept. in the "Specifications and acceptance Parameters for Supplied Products". They are also subject to an initial audit by the Mgr. Procurement. First lot acceptance is based on tighter criteria. Once a supplier is accepted, they are placed on the ASL with a "probation" rating. Following 10 good shipments without any supplier corrective action requests (SCARs), the "probation" indication is removed.
As part of the supplier agreement, suppliers understand that they are subject to ongoing monitoring. This is done by collecting and analysing data on the quality and timeliness of the shipments. Suppliers are also subject to periodic re-audits - in particular to follow-up on SCARs. Poor performance (3 consecutive bad lots or 4 SCARs in one year) will re-instate probation. Three supplier evaluation files were reviewed.
The process for purchasing goods and services is described in QP-56 "Purchasing". Purchasing is based on production plans and schedules set by the VP Operations. The standard PO form used by the company requires sign-off by the Mgr. Procurement confirming that the supplier is on the ASL and that the appropriate specifications are identified on the PO.
The reception of goods is controlled by QP-34 and by the Purchased Material Approval procedure (QP-57). A receiving technician, <Name>, explained the process and demonstrated it with lot 22-3 of <supplied product 2>. The inspection and acceptance of the supplied product is based on the criteria identified in the project plan. Final release is authorised by the Mgr. Procurement.
When supplied goods or services do not conform to requirements, a SCAR is issued by the Mgr. Procurement. These are also forwarded to the Material Review Board (MRB) to anticipate any production issues and to close out the SCAR (see QP-58). Several SCAR's were reviewed. The company is diligent in following-up on SCARs and ensuring that problems are resolved. Goods can only be accepted on concession by authority of the MRB.
|Area Visited||Procurement, Receiving, Warehouse|
|Persons Interviewed||<Name> Mgr. Procurement
<Name> Receiving Technician
|Purchasing QP-56 R3
Purchased Material Approval QP-57 R2
Receiving QP-34 R5
Supplier Approval QP-57 R2
Supplier Monitoring SOP-3325
ASL R77 (Feb.6 2011)
SCAR QP-58 R1
Project Quality Plan for <Product 2> R1 - "App. C Specifications and acceptance Parameters for Supplied Products"
|Product/Components Reviewed||Supplier evaluations for <Supplier Product 1>, <Supplied Product 2>, <Supplied Service 1>
SCAR #45, 78, 79, 87
Inspection record for Lot 22-3 for <Supplied Product 2>
|Conclusions||Purchasing activities are well defined and controlled. Appropriate authorities are defined. The audited activities give confidence that supplied products conform to requirements. The Company is in conformity with 7.4|
Audit Summaries should also include the following where applicable:
b) Description of Major Changes
When the activity or process being audited has been subject to a major change, this should be described in the audit report. This includes major changes to products or processes, changes to the organizational structure or ownership, as well as changes to key personnel and facilities and to the QMS as a whole. The description of these changes should include a discussion of their relevance and impact on regulatory requirements and submissions to regulators. The description can be included in the audit summaries or under a separate heading.
Major changes are changes that have the potential to affect the conformity of the product with specified requirements or the ability of the QMS to conform to requirements or to meet quality objectives. In practical terms, major changes are those that have the potential to affect the safety or effectiveness of the medical devices or to affect the manufacturer's ability to comply with regulatory requirements.
Major changes include changes to the following:
- organisational structure;
- key personnel;
Although design changes may be subject to licence amendment (CMDR 34) and other changes subject to annual reporting (cf. CMDR 43(1)), not all major changes are subject to regulatory scrutiny. Therefore, as an important part of effective ongoing oversight, major changes should be addressed in audit reports.
Major changes should be described in the report including their relevance and impact, particularly with respect to applicable regulatory requirements. Discussions of major changes should include information on whether the manufacture has made a regulatory submission or, where no regulatory submission has been made, that the manufacture has considered the need for a submission and has a documented rationale for not proceeding with a regulatory submission. These discussions should also address the ongoing suitability of the QMS in light of the changes.
Major changes should ideally be discussed in context in the audit summaries although this can also be done under a separate heading.
A. <Company> has recently updated the software of the device from version 83.6 to 84. The changes were made to address internal coding standards and nomenclature inconsistencies. As part of design review, the Engineering department concluded that this change did not affect the form or function of the device and therefore did not require a notification to any regulatory agency.
B. <Company> has undertaken a design change project to its balloon dilatation catheter <Model Number>. The company is in the process of validating new packaging and qualifying a new contract sterilizer for the devices. Company officials have stated that no licence amendment is planned for this device. A nonconformity was issued (see NC-01) as this is a significant change to a class IV medical device and requires a medical device licence amendment (CMDR 34).
C. A second plasma welder line has been qualified and is in the process of validation (as per Master Validation Plan OP-34-2) for the CV snare production line. The new unit is identical to the existing one. No production units have been manufactured using the new welder. The RA manager has indicated that no submission to regulatory authorities will be necessary because of this change.
D. The Engineering department has been separated into two separate departments. <Name>, the former VP of Engineering is now the VP of Research and Development and is responsible for design controls. <Name2> is now VP of Production (this was a responsibility of the former Engineering Dept.) and is responsible for design transfers and production activities (excluding support functions that report to the VP Facilities). These changes have no regulatory impact. The new authorities have been recorded in the amended QMS documentation.
Identification of any information that was requested and refused by the auditee should be included in the report. This includes refusal of access. Any other obstacles encountered that have the potential to impact the validity of the audit conclusions should be identified in the audit report.
Alternatively, these obstacles can be described in section 2.3.4 d) - Reliability of Audit.
As part of fair presentation, the audit report should include any obstacles to the audit that were encountered. Obstacles include both deliberate interference with the audit team and situations that hamper the audit. Deliberate actions include:
- refusing (or "forgetting" ) to provide certain documents or records;
- refusing to answer certain questions or providing evasive answers;
- refusing access to certain areas;
- wilfully hampering or delaying the auditors;
- being generally uncooperative or combative.
Situations that may be encountered that have the potential to affect the validity of the audit conclusions include:
- line shutdowns;
- absence of key individuals;
- certain processes or activities not being performed during the audit;
- power outages;
Obstacles should be reported in the audit summaries where they are encountered. Alternatively, obstacles of a general nature can be reported in the section outlining factors affecting the reliability of the audit in the 'conclusions' portion of the report (see below).
d) Follow-up on Past Nonconformities
Where the implementation of correction and corrective actions stemming from past nonconformities is verified, this verification should be included in the audit report, either as part of the Audit Summaries section or under a separate heading. If nonconformities from past audits cannot be closed, this should be indicated.
The follow-up of past nonconformities including verification of correction and corrective action often forms part of the audit objectives. As such, this part of the audit must be recorded in the audit report regardless of any additional pro forma documents used. Reporting on this activity can occur within the audit summaries or under a separate heading in the 'Audit Findings' portion of the report.
If past nonconformities cannot be closed, the report should indicate this. Registrars should escalate the conditions placed on the manufacturer and its registration in such cases.
Registrars are free to use separate nonconformity reports or forms, however the audit report should include, for each nonconformity: a statement of nonconformity; the criterion against which the nonconformity is raised; and the supporting objective evidence. These items should be put into context and included in the appropriate audit summaries. This does not preclude further reporting on nonconformities in the report or elsewhere.
Any unresolved objections by the manufacturer to the issued nonconformities should be recorded.
Where the manufacturer undertakes cause analysis, correction or corrective action before the end of the audit, a mention of this may be made in the report, however it does not eliminate the need to report the nonconformity.
Nonconformities and their discussion should be included in the audit summaries in order to provide context for the findings. Since they constitute an important class of audit finings, nonconformities should be supported by an adequate amount of evidence in the audit report.
When including nonconformities in the audit report, authors should be mindful to include all three of the following:
- the statement of nonconformity, clearly highlighting the non-conforming situation;
- the requirement that is not met; and,
- the evidence to support the finding of non-conformity.
By including these three items in context (i.e. in the audit summary), the report authors ensure that users of audit reports (certification bodies, regulators) will understand the finding as it was uncovered and be better able to gauge the significance of the nonconformity.
When the manufacturer disagrees with a finding or its classification, and the disagreement cannot be resolved, auditors should note the objection in the audit report.
While it is generally not a recommend practice, some auditees attempt to undertake correction or corrective action in response to nonconformities before the end of the audit. When this occurs, it may be noted in the audit report, but this should not affect the reporting of the nonconformity in any other way.
f) Areas Not Audited
When areas within the scope of the audit (as defined in the audit plan) are not audited or not sufficiently covered, this should be noted in the audit report.
The audit report should highlight any area within the planned scope of the audit that was not covered. It may be useful to include the reason(s) for which the area was not audited. Report authors can also make reference to these un-audited areas when discussing the achievement of audit objectives and the reliability of the audit conclusions.
The audit report should provide clear conclusions about both the conduct of the audit and its overall outcome and results. Conclusions provided in this section of the report should relate to the quality management system as a whole.
The report should include conclusions regarding the overall conformity and effectiveness of the management system. These conclusions should reflect the audit team's evaluation and synthesis of all the audit evidence collected and the related findings. In particular, conclusions should be presented in relation to each audit objective described earlier in the audit report. These overall conclusions should flow naturally from the findings presented in the audit report. They should be brief, informative, unambiguous, and accurate.
This section of the report should also address technical matters related to the overall conduct of the audit. In particular, report authors should indicate whether all audit objectives have been accomplished as planned and should report any factors affecting the reliability of the audit.
Recommendations to the certification body by the audit team should also be recorded in the report. These recommendations cover a range of items, from certification, to audit programming, including necessary follow-up actions for both the registrar and the manufacturer.
11.2 Specific requirements
The following requirements can be found in section 2.3.4 of GD211.
a) Conformity with Audit Criteria
A brief summary and conclusion regarding the conformity of the QMS as it is implemented with each set of audit criteria in 2.3.2 b) above should be included in the report. The conclusions should be unambiguous as to the conformity or nonconformity of the QMS.
The report should include a conclusion regarding the conformity or non-conformity of the QMS with each set of audit criteria identified in 'Information about the Audit'. The conclusions should be clear as to the conformity status of the QMS.
Conclusions should not be contradictory to any of the findings in the report. They should not be ambiguous and they should not have double-meaning (e.g. "notwithstanding the identified major deficiencies, the QMS conforms to the audit criteria").
In situations where there are multiple sets of audit criteria, such as during a combined audit, it is a best practice to use a separate conclusion for each set of audit criteria.
A. Based on the interviews and evidence observed, it is concluded that the system is effectively implemented and remains in conformity with ISO 13485:2003. The identified nonconformities are deemed to be minimally significant and do not affect the finding of overall conformity.
B. As indicated by the individual findings of conformity in the preceding audit summaries and the absence of any identified nonconformities, the system is found to conform as a whole to the requirements of ISO 13485:2003.
C. During the audit, the company was not able to provide sufficient evidence of conformity to numerous requirements of the audit criteria (see findings above for details). Therefore, the audit team concludes that the QMS of the manufacturer is not in conformity with ISO 13485:2003.
D. Considering the number of minor nonconformities identified during this audit, taken together with the conformity history of the manufacturer, the audit team concludes that the QMS is not fully in conformity with ISO 13485:2003.
E. The identification of three (3) major nonconformities and the premature termination of the audit preclude the audit team from concluding on the conformity of the QMS with the audit criteria.
The report should include a brief summary and conclusion regarding the effectiveness of the QMS in meeting quality objectives. One of these quality objectives includes compliance with applicable regulatory requirements.
The report should include a conclusion regarding the effectiveness of the QMS in meeting quality objectives. These conclusions should be based on both the manufacturer's determination of QMS effectiveness during management review and the objective evidence of effectiveness (or lack thereof) collected by the audit team.
For regulators, the most important quality objective is compliance with regulatory requirements. Conclusions on the effectiveness of the QMS should indicate whether the manufacturer complies with regulatory requirements.
If the audit criteria included more than one set of regulatory requirements, the conclusion of effectiveness should address the ability of the QMS to ensure compliance with each set of regulatory requirements separately.
A. The quality management system is judged to be effective in allowing the company to meet its business and quality objectives and to ensure compliance to the Canadian MDR. Evidence indicates that business targets are met. Internal quality standards are met.
B. The audit evidence reviewed clearly demonstrated that the company is meeting its quality objectives as set by management. The QMS is mature and the company's quality culture leads to a high level of implementation. Regulatory processes are well implanted and the company has been diligent in addressing all regulatory requirements identified in the QMS. The audit team concludes that the QMS is effective.
C. Based on the evidence reviewed during the audit, the audit team cannot conclude that the QMS is effective in allowing the manufacturer to meet its quality objectives and comply with applicable regulatory requirements. In particular, several situations were uncovered (see issued nonconformities) where the manufacturer had failed to address regulatory requirements.
D. The company's QMS is well adapted to the business environment of <Company>. Quality objectives are generally met or exceeded. Management is proactive in addressing any objectives that are not met. The requirements of the MDD 93/42/EEC are adhered to. The QMS is generally effective in ensuring compliance to the Canadian MDR, however the post-market activities related to reporting incidents are designed to address European requirements and do not ensure that certain types of incidents will be reported to Health Canada as required by the MDR (see NC# 2).
c) Confirmation of Audit Objectives
The report should confirm that all audit objectives in 2.3.2 c) have been met. Where any of the audit objectives have not been met, an explanation should be provided.
The report should confirm that all audit objectives, as stated previously in the report, have been achieved. When an objective is not met or only partially completed, the audit report should indicate this and outline the reason(s).
The principle of fair presentation outlined above should guide report authors when disclosing unmet objectives and the reasons behind these situations. Disclosing unmet objectives prevents unwarranted conclusions from being drawn by report users. This practice also allows certification bodies to appropriately plan future oversight by outlining areas requiring additional focus or resources as well as providing the impetus for adjustments to the audit programme.
A. All audit objectives as stated above were completed. The audit was executed as planned.
B. It was not possible to complete all audit objectives. The audit team was unable to assess the effectiveness of the QMS in ensuring compliance to Part 1 of the CMDR since the manufacturer has not licensed any devices in Canada and therefore has not implemented any of the CMDR requirements fully. All other audit objectives were completed as planned.
C. The audit was prematurely terminated due to a high number of major nonconformities and an imminent risk to public health. The audit objectives were therefore not achieved.
D. It was not possible to complete all audit objectives as planned. The requested extension to scope could not be fully investigated because the manufacturer has not yet completed the design transfer activities for the paediatric vascular access port.
d) Reliability of Audit
The report should outline any factors encountered that may decrease the reliability of the audit. This may include such factors as a shortfall in auditor time, the absence of a needed technical competence, or any obstacle not mentioned under 2.3.3 c).
Once again, the principle of fair presentation warrants full disclosure of any factors that could affect the reliability of the audit findings or conclusions. Such factors come in many forms and could include:
- a shortfall in auditor time (due to a variety of reasons);
- the absence of a technical competence needed to evaluate a special process or technology;
- the absence of a key manager or employee;
- the unavailability of certain records;
- the unavailability of representative samples (due to, for example, new activities or processes).
Disclosing the factors that can affect the reliability of the audit findings and conclusions allows the users of reports to make informed judgements based on the results of the audit and only serves to increase the confidence in the conformity assessment performed.
A. No factors were encountered that could reduce the reliability of the audit or its conclusions.
B. The only factor possibly affecting the reliability of the audit was the absence of the Human Resources manager. Subordinate staff attempted to answer questions and provide information in his stead, but not all aspects of resources management were thoroughly investigated as a result.
C. The audit team had to rely on ad hoc Tagalog interpretation from production staff in order to interview certain operators.
D. Please note the audit team only had summary experience with software. Therefore, the control software validation portion of the design file was only briefly reviewed.
E. Because the audit team opted to investigate the outsourcing of certain manufacturing steps previously done in-house, audit time allocated for the audit of resource management and training was reduced by 60% leading to a significant reduction in the number of records sampled.
The audit team's recommendations should be included in the report. Recommendations should be made with regards to:
- any follow-up actions by the registrar, changes to the audit programme, or changes to the number of auditor-days; and,
- the initial or continuing certification of the quality management system, together with any conditions or observations.
The audit team should recommend any necessary follow-up actions for the registrar. These could include specific areas requiring additional focus during the next audit, potential audit trails, or additional oversight activities (e.g. off-site document review). The audit report should also contain any recommendations for changes to the audit programme (e.g. type, number, or frequency of audits), changes to the on-site audit time, or changes to the composition of the audit team (particularly with respect to technical competence).
When making recommendations to the certification body regarding initial or continuing certification, audit teams should also include any appropriate conditions or observations attached to the recommendation.
A. Given that the audit objectives have been accomplished with no obstacles and that the QMS has been found to be in conformity with the audit criteria and to be effective, the audit team recommends to the certification body that the certification of <company> to ISO 13485:2003 under CMDCAS be maintained. No additions or modifications to the audit programme are suggested.
B. The audit team recommends that <manufacturer> be considered for certification to ISO 13485:2003 under CMDCAS once suitable plans for corrective action have been accepted for the two minor nonconformities issued. Furthermore, it is recommended that the manufacturer undergoes its first surveillance audit in the first 6 months following certification in order to verify the implementation of corrective action and to validate the findings of this audit since this company is a start-up and has not fully implemented its production processes yet.
C. The lead auditor recommends on the basis of this assessment the ongoing certification of <Company>. It is recommended that an extra one person-day on-site be added to the audit programme for next year's recertification considering the nature of the findings during this certification cycle as well as the changes undergone by the manufacturer.
D. The audit team recommends the immediate suspension of the certification of <Company> until an on-site verification of the correction and corrective action of the four identified major nonconformities can be performed. <Certification Body> is urged to include specific competencies in the audit team dealing with sterilization and packaging issues given the observed conditions and findings listed above.
12.0 Identification and dating
The identification of the author or authors of the report is important in order to provide points of contact should the report require clarification.
Because auditing is time sensitive, report authors and registrars are encourage to employ appropriate dating and version control practices. Such practices allow for timeline reconstruction and retrospective investigation.
Great care should be taken to avoid confusion when dealing with multiple versions of reports. When users of reports cannot reconstruct the reporting history and timeline nor identify the scope and nature of changes between versions, reports lose much of their value.
12.2 Specific requirements
The final audit report should include the name(s) of the author(s) of the report. The report should also be dated on its final date of issue and include version control information where necessary.
The dating of reports should use a dating convention that is not subject to confusion (see e) Audit Dates above.) If reports undergo editing or correction, or where multiple versions of reports exist, suitable version control information should be added.
12.3 Additional Resources
The following additional resources are available:
ISO/IEC 17021:2006 - Conformity assessment - Requirements for bodies providing audit and certification of management systems
ISO 19011:2002 Guidelines for quality and/or environmental management systems auditing
GD207: Guidance on the content of ISO 13485 Quality Management System Certificates Issued by Health Canada Recognized Registrars
GD210: ISO 13485:2003 Quality Management System Audits Performed by Health Canada Recognized Registrars
GD211: Guidance on the content of quality management system audit reports
SG4/N33R16:2007 Guidelines for Regulatory Auditing of Quality Management Systems of Medical Device Manufacturers - Part 3: Regulatory Audit Reports
Report a problem or mistake on this page
- Date modified: