Overview of the immunization product quality control practices of vaccinators in Montérégie

Canada Communicable Disease Report

1 January 2007

Volume 33

Number 01

Blackburn M, MD, MSc, FRCPC (1, 2), Bellemare S, Medical student (2), Gagné G, Medical student (2), Gagnon J, Medical student (2), Gagnon-Gervais K, Medical student (2), Gaudet C, Medical student (2), Messier Fugère L, BSc inf (1)

  1. Agence de la Santé et des Services sociaux de la Montérégie, Direction de la santé
    publique (Quebec)

  2. Faculté de médecine, Université de Sherbrooke (Quebec)

Introduction

In order to ensure optimum vaccine efficacy, a number of standards must be met. Vaccines must be maintained at a temperature between 2o C and 8o C, in accordance with the cold chain system. Any break in the cold chain can have multiple consequences. Some individuals who are given a product exposed to a break in the cold chain may remain susceptible to contracting the infection in question1,2, while being lulled into believing that they are protected. Moreover, breaks in the cold chain generate additional costs, including the cost of caring for vaccine recipients who subsequently become infected, as well as the costs associated with withdrawing and replacing compromised immunization products.

Good practices with respect to immunization product storage and transportation are set out in the Protocole d'immunisation du Québec (PIQ) and the Guide des normes et pratiques de gestion des vaccins, two documents distributed to all vaccinators in Quebec 1,3. Vaccinators can also consult the Canadian immunization guide2. Here, in brief, are the key recommendations set out in these documents:

  • All refrigerators containing vaccines should be serviced annually (door seal checks, coil cleaning, thermostat checks).
  • Refrigerators should be used exclusively to store vaccines. No food, drink or laboratory specimens should ever be stored with immunization products, in order to minimize the frequency with which refrigerator doors are opened.
  • A “DO NOT UNPLUG” sign should be posted next to every wall outlet to which a refrigerator is connected.
  • Vaccines must be stored in the original boxes provided by the manufacturer.
  • There should be sufficient space around the vaccines to allow air to circulate.
  • Vaccines should be stored in the middle of the refrigerator. They should never be placed on the door shelves.
  • There should be water bottles or unfrozen cold accumulators (ice packs) in the upper and lower sections of the refrigerator, as well as in the door compartments, in order to stabilize the temperature.
  • Vaccines should only be removed from the refrigerator immediately before administration.
  • Every refrigerator must be equipped with a maximumminimum thermometer.
  • The thermometer probe should be placed in the middle of the refrigerator, where the vaccine bottles are grouped together.
  • Maximum and minimum refrigerator temperatures should be read twice daily and recorded onto a temperature chart.
  • These charts should be retained for at least 4 years.
  • After each reading, thermometers must be reset to zero.
  • Products exposed to temperatures below 2o C or above 8o C should be quarantined and labelled “DO NOT USE.” They should not be discarded. The Direction régionale de la santé publique should be contacted so that it can assess the situation and make appropriate recommendations.

The primary goal of the study was to develop an overview of how quality control standards for immunization products are enforced by vaccinators in private medical clinics and in CHSLDs (residential and long-term care facilities) in Montérégie, a region in southwestern Quebec that is home to 1.3 million inhabitants. Vaccines are administered in Montérégie by physicians in private medical clinics and by nurses and physicians in health facilities. In February 2006, the region had 211 vaccinators in private clinics and 34 vaccinators in CHSLDs. This article reviews the results of our survey of these 245 vaccinators.

Method

Of the 245 vaccinators in the region, 24 were excluded from the study because they had stopped practising or could not be reached by fax. The survey of the remaining 221 eligible vaccinators was carried out with the aid of a self-administered questionnaire, which was sent to every vaccinator by fax. Vaccinators were also asked to forward a copy of their temperature charts for the month of February for every refrigerator in which vaccines were stored during that period. Vaccinators had 48 hours to return their completed questionnaire and the requested charts. Within 2 days of the questionnaires being sent out, the response rate had reached 31%. Telephone follow-up of non-respondents ultimately produced a response rate of 57% (127 respondents).

Following the data-gathering process, four private clinic vaccinators from among those who had agreed to receive the visit of two medical students were visited for on-site validation of questionnaire information. Finally, all the forwarded temperature charts were examined. Detected cold chain breaks were compared with those previously reported to the Direction régionale de la santé publique in order to estimate the number of cold chain breaks that had gone undetected or unreported.

For the purposes of data analysis, frequency tables were established for each questionnaire variable. Four composite variables were created by combining several questionnaire elements. These variables were used to establish the percentage of respondents whose practices could be deemed compliant with a set of standards, based on the four definitions developed for this study (Table 1). The main differences among the four definitions had to do with the presence or absence of other pharmaceutical products in refrigerators, the frequency of temperature readings and the retention or non-retention of temperature charts for ≥ 4 years.

Chi-square or Fisher tests were carried out to determine whether the number of refrigerators was associated with compliance with quality standards. Data compilation and analysis were performed using SPSS 13.0 software.

Table 1. Description of the elements included in the four definitions of compliance

Standard

Definition A

Definition B

Definition C

Definition D

Use of refrigerators to store immunization products

Refrigerators used to store immunization products only

Refrigerators used to store immunization products only

Refrigerators used to store pharmaceutical products only †

Refrigerators used to store pharmaceutical products only †

Thermometer in every refrigerator

YES

YES

YES

YES

Adequate reading of thermometers §

YES

YES

YES

YES

Frequency of refrigerator temperature readings

Twice daily

Twice daily

Once a day

Once a day

Temperatures recorded onto a monthly chart

YES

YES

YES

YES

Monthly charts retained for ≥ 4 years

YES

Not considered

YES

Not
considered

Thermometers reset to zero after every reading

YES

YES

YES

YES

† Including immunization products
§ Reported-reading maximum and minimum temperatures indicated on the maximum-minimum thermometer display.

Results

Of the 127 respondents, 84% were from private medical clinics (n = 107) and 16% were from CHSLDs (n = 20). Some 80% of respondents stated that they used only one refrigerator to store immunization products. Table 2 shows the results with respect to the reported practices of respondents. Most perform daily (21%) or twice daily (71%) readings of maximum and minimum thermometer temperatures. Moreover, 84% of respondents systematically (72%) or frequently (12%) record these temperatures on charts provided for this purpose. Among respondents, 85% retain these charts, with 52% retaining them for ≥ 4 years and one third (32%) retaining them for < 2 years. Thirty-one percent of respondents indicated that they did not possess a guide on the management of immunization products.

Table 2. Percentage of respondentswho enforce the various quality control standards for immunization products (n = 127)

Standard

Standard enforced according to respondent

yes
n (%)

no
n (%)

don't know
n (%)

Condition of refrigerators checked annually

51 (40)

48 (38)

28 (22)

A “DO NOT UNPLUG” sign is posted next to every wall outlet to which a refrigerator is connected

61 (48)

61 (48)

5 (4)

Immunization products stored on the centre shelf of the refrigerator

51 (40)

75 (59)

1 (1)

Immunization products stored in the middle of the refrigerator ¶

118 (93)

9 (7)

0 (0)

Immunization products always stored in the box provided by the manufacturer *

124 (98)

3 (2)

0 (0)

Space left between immunization product boxes

111 (87)

14 (11)

2 (2)

Refrigerators used exclusively to store immunization products

19 (15)

82 (65)

26 (20)

Refrigerators used exclusively to store pharmaceutical products †

59 (47)

66 (52)

2 (1)

Presence of water bottles or cold accumulators (ice packs) in refrigerator

66 (52)

60 (47)

1 (1)

Thermometer in every refrigerator

123 (96)

2 (2)

2 (2)

Maximum-minimum thermometer probe located in the right place ‡

90 (71)

34 (27)

3 (2)

Adequate reading of thermometers §

98 (77)

14 (11)

15 (12)

Thermometers reset to zero after every reading

103 (81)

20 (16)

4 (3)

¶ Immunization products located at the centre of either refrigerator shelf, but not pressed against the sides of the refrigerator.
* Always or often
† Including immunization products
‡ Probe located at the centre of the cluster of immunization product bottles
§ Reported-reading maximum and minimum temperatures indicated on the maximum-minimum thermometer display.

According to the definitions developed for this study, between 5% and 28% of respondents have practices that are consistent with the entire range of quality control standards for immunization products (Table 3). No statistically significant difference was observed between the number of refrigerators used to store immunization products and the quality control practices in place for these products.

Table 3: Percentage of respondents who reported having compliant practices, based on the various definitions of compliance (n = 127)

Definition §

Compliant n (%)

Non-compliant n (%)

Definition A

6 (5)

121 (95)

Definition B

11 (9)

116 (91)

Definition C

22 (17)

105 (83)

Definition D

36 (28)

91 (72)

§ See Table 1 for a description of the definitions developed for this study.

Respondents forwarded 99 monthly temperature charts. An analysis of these charts revealed that 13 cold chain breaks had occurred during the first 3 weeks of February 2006. None of these breaks had been previously reported to the Direction régionale de la santé publique.

Visits were made to four private medical clinics a few days after data collection had been completed. Observations made at these times showed that the information provided in the questionnaires had been accurate.

Discussion

When each standard is considered on its own, Table 2 reveals that they are being applied by more than 40% of vaccinators. However, when several standards need to be applied simultaneously, according to six or seven criteria, very few vaccinators demonstrate practices that can be deemed compliant. Still, a number of biases may have influenced these results. Most notably, a selection bias is probably present in this study. Although the response rate may seem low, it is in fact relatively satisfactory given the short amount of time provided to participate (48 hours), as well as the target population, which was primarily composed of physicians and nurses in private practice. However, it is also highly likely that the respondents were not representative of the targeted vaccinator population. We must consider the possibility that vaccinators who were less compliant in their practices were also less likely to participate. An information bias is also present, since the information was based on statements made by the respondents themselves. Although subsequent observation served to corroborate the statements of four of the respondents, this level of validation is fragmentary at best. Respondents' self-reported practices may well be more in line with recommendations than are their actual practices. In fall 2005, a letter was sent to the CHSLDs offering to provide them with a free thermometer if one was needed. This may have helped to improve the practices of these centres, but it may also have increased the desirability bias mentioned earlier. Given these biases, the number of vaccinators whose practices are consistent with the recommendations may in fact be lower than our estimates suggest.

The large number of monthly temperature charts forwarded by respondents constitutes an interesting result in itself. For one thing, vaccinators appear to be receptive to such an approach. These charts provide information that is less vulnerable to various biases by virtue of the fact that it was collected before the survey and was not self-reported, although the decision to forward the charts was clearly voluntary. The number of cold chain breaks detected in this study - 13 breaks among 127 respondents - seems quite high. None of these breaks had been reported to the Direction régionale de la santé publique; moreover, the study does not provide the means to determine the extent to which these breaks were actually identified as such by the vaccinators prior to the survey. These results suggest that regional and provincial statistics on immunization products that are subjected to inadequate temperatures - statistics that are based on the statements of vaccinators - may in fact underestimate the scale of the problem. In Montérégie, it is estimated that a little less than 2% of immunization product doses were lost as a result of breaks in the cold chain in 2004-2005 (Messier Fugère L., Immunization product officer, Direction de la santé publique de la Montérégie, personal communication, 2006).

It is difficult to estimate the extent to which the results of this study can be generalized to all vaccinators in Quebec or Canada. Several factors can influence local or regional practices, including the dissemination of clear recommendations by a recognized organization, the availability of the guides and tools needed to effectively monitor immunization products, the availability of training, or the practice profiles of vaccinators. Several facilitating factors were already present in the environment of Montérégie vaccinators prior to the study. Between 1999 and 2001, for example, the Ministère de la Santé et des Services sociaux du Québec pursued an initiative designed to assist local and regional custodians of immunization products in the implementation of provincial standards on the handling of these products4. As part of this initiative, a concise vaccinator's reference guide on immunization product quality control procedures was produced in partnership with the firm Aventis Pasteur3. The guide was distributed for free to all vaccinators in Montérégie, along with a maximum-minimum thermometer, a supply of monthly refrigerator temperature charts, a refrigerator magnet reminder, a “DO NOT UNPLUG” sticker, and all the materials needed to properly transport immunization products. A subsequent evaluation of this initiative showed that marked improvements had taken place across the province in terms of the number of refrigerators equipped with a thermometer, as well as temperature-reading practices, and the retention of temperature charts4. It should be stated, however, that the initiative was still underway in Montérégie when the evaluation was performed, and that only 46% of medical clinics had received the material by that time (Messier Fugère L., Immunization product officer, Direction de la santé publique de la Montérégie, personal communication, 2006). Of the 188 private clinic vaccinators who took part in our study, 70% ultimately received the material during an individual visit, while 30% received it by courier delivery, which was followed, in some cases, by a telephone call 1 week later. The latter method was also employed for CHSLDs.

Despite the effort that went into this initiative, a variety of factors may lessen the benefits that can be derived from it in the long term. Indeed, staff rotation and the assignment of duties in medical clinics and facilities can result in individuals assuming responsibility for immunization products without having received appropriate training. What is more, this kind of initiative does not provide an opportunity to follow up with vaccinators in order to review the enforcement of standards with them. Such follow-up contact can be most effective when practices are modified through the addition of new vaccines or when medical clinics welcome new clienteles5.

The identification of a cold chain break can provide a useful opportunity to review work methods with vaccinators, as well as to identify problems and potential solutions. Given the consequences of cold chain breaks (patient recalls, return of exposed products, etc.), vaccinators may be more receptive to advice at these times and more motivated to implement the necessary corrective measures. In order for such follow-up to take place, however, it is important that the vast majority of vaccinators complete their monthly temperature charts. The results of this study suggest that it may also be necessary to forward these charts to the Direction régionale de la santé publique in order to improve the detection of cold chain breaks. The Direction de la santé publique de la Montérégie is currently examining the feasibility of such an approach.

Conclusion

Although modest in scope, this study showed that a significant number of vaccinators were not in compliance with certain quality control standards for immunization products. The efficacy of vaccination programs is dependent upon the administration of an immunization product to the largest possible number of persons within a target population. Access (geographic, financial, cultural) to vaccines constitutes an important determinant of vaccination coverage2. Therefore, it is important to be able to offer vaccination in a variety of contexts and through a variety of care providers in order to protect the largest possible number of people2. Increasing the number of vaccination opportunities also means that an increasing number of sites will need to maintain the highest quality control standards for immunization products. Efforts to make immunization products more accessible must not result in a loss of efficacy in the products being administered. The study reviewed in these pages suggests that we must remain vigilant with regard to the quality of immunization products, in order to ensure that the effectiveness of vaccination programs is maintained.

Acknowledgements

The authors wish to thank all those who participated in the study, as well as the following individuals for their expertise and contribution to the survey: A-M. Clouâtre, Direction de la santé publique de la Montérégie, Longueuil (Quebec), M. Villeneuve, Direction de la santé publique de la Montérégie, Longueuil (Quebec).

References

  1. Comité sur l'immunisation du Québec. Ministère de la Santé et des Services sociaux du Québec. Protocole d'immunisation. Quebec: Ministère de la Santé et des Services sociaux du Québec, 2004;69-77.

  2. National Advisory Committee on Immunization (NACI). National guidelines for childhood immunization practices. CCDR 1997;23(ACS-6):1-6.

  3. Alain L. Guide des normes et pratiques de gestion des vaccins à l'intention des
    vaccinateurs médecins et infirmières
    . Québec: Direction générale de la santé publique, Ministère de la Santé et des Services sociaux du Québec, 1999.

  4. St-Onge L. Contrôle de la qualité des produits immunisants distribués dans le cadre du programme québécois d'immunisation. Québec: Direction de la protection de la santé publique, Ministère de la Santé et des Services sociaux du Québec, 2003.

  5. Koutsavlis T. Disseminating practice guidelines. Institut national de santé publique du Québec, 2001.

Page details

Date modified: