Testing point-of-care test kits for the identification of infectious syphilis

CCDR

Volume 48-2/3, February/March 2022: Syphilis Resurgence in Canada

Surveillance

Laboratory evaluation of two point-of-care test kits for the identification of infectious syphilis

Raymond SW Tsang1, Michelle Shuel1, Kristy Hayden1, Paul Van Caeseele2, Derek Stein2,3

Affiliations

1 Syphilis Diagnostic Unit, National Microbiology Laboratory, Public Health Agency of Canada, Winnipeg, MB

2 Cadham Provincial Public Health Laboratory, Winnipeg, MB

3 Department of Medical Microbiology and Infectious Diseases, University of Manitoba, Winnipeg, MB

Correspondence

raymond.tsang@phac-aspc.gc.ca

Suggested citation

Tsang RSW, Shuel M, Hayden K, Van Caeseele P, Stein DR. Laboratory evaluation of two point-of-care test kits for the identification of infectious syphilis. Can Commun Dis Rep 2022;48(2/3):83–8. https://doi.org/10.14745/ccdr.v48i23a05

Keywords: infectious syphilis, point-of-care tests

Abstract

Background: Syphilis is a sexually transmitted disease that can have atypical clinical presentations. Conventional laboratory tests to confirm the diagnosis are not rapid enough to affect clinical decision on treatment and contact tracing. Rapid point-of-care tests (POCT) can be useful for control of infectious diseases; however, no POCT for syphilis detection is currently available in Canada. The aim of this study is to evaluate two POCTs (RevealTM Rapid TP (Treponema pallidum) Antibody test and DPP® Syphilis Screen and Confirm test) for detection of infectious syphilis.

Methods: One hundred serum samples with known syphilis serological status, based on treponemal and non-treponemal test results, were analysed in the laboratory with two POCTs by two independent operators in a blind fashion. Results were analysed to evaluate their ability to detect infectious syphilis.

Results: The Reveal Rapid TP Antibody POCT showed an overall sensitivity of 95.0% and a specificity of 83.3%, while the DPP Syphilis Screen and Confirm POCT showed a sensitivity of 87.5% and a specificity of 98.3%. Both POCTs gave a sensitivity of 100% on active syphilis samples with Venereal Disease Research Laboratory (VDRL) titres of greater than 1:4, but their sensitivities decreased for samples with low VDRL titres. Both POCTs gave weakly or very weakly reactive results on 11.3%–25.0% of the treponemal antibody positive samples.

Conclusion: This laboratory evaluation has shown promising results for both POCTs to detect infectious syphilis. Further evaluations in the field would be required to confirm this preliminary finding.

Introduction

Infectious syphilis has been on the rise in North America since the early 2000sFootnote 1Footnote 2. In recent years, the increase has been substantial: in Canada, the rate of infectious syphilis has increased about 3.4-fold within 10 years from just under five cases per 100,000 population in 2009 to about 17 cases per 100,000 population in 2018Footnote 3. Also, a shift in increasing rates of infectious syphilis in females of reproductive age (15 to 39 years) has been observed in both the United States (US) and CanadaFootnote 2Footnote 4, leading to an increase in congenital syphilis in both countries. The reasons behind the increase in syphilis cases in Canada have been discussed recentlyFootnote 5, and may include better access to testing, increased sensitivity of enzyme immunoassays employed in the screening of syphilis, lapse in the practice of safe sex and changes in the social behaviour coupled with availability of social media platforms to facilitate recreational sexual encounters. To combat this increase, education to increase better awareness of syphilis, more timely and in-depth surveillance data to allow interventions to be developed that can target at risk behaviours or different at-risk ethnic groups, and better testing methods that can provide immediate results to allow for earlier treatment and contact tracing, have also been suggestedFootnote 6.

Syphilis has been described as a great imitatorFootnote 7; clinical and laboratory diagnosis of infectious syphilis can be challenging. The most rapid and confirmatory diagnostic test for infectious syphilis is dark-field microscopy, which looks for motile spirochetes in clinical specimens, but this method is now almost non-existent in Canada due to the required technical expertise and the relative insensitivity of the method. Conventionally, syphilis is diagnosed serologically by measuring antibodies to both treponemal antigens (e.g. using chemiluminescent microparticle immunoassay, CMIA, which detects IgG and IgM antibodies) and non-treponemal antigens (e.g. using either rapid plasmin reagin, RPR, or Venereal Disease Research Laboratory, VDRL, tests). Patients with active or infectious syphilis show positive results with both the treponemal and non-treponemal tests while patients with past syphilis show positive results only with the treponemal test. Although these conventional serological tests and the newer molecular polymerase chain reaction diagnostic methods are accurate and sensitive, the results are not available rapidly enough to allow clinical decisions on both treatment of patients and contact tracing. Furthermore, there are currently no licensed commercial nucleic acid amplification tests for syphilis on the market.

While point-of-care tests (POCTs) for syphilis have been developed in the early 2000s and have been in use in a number of resource limited countries that lack the laboratory infrastructure, equipment and qualified personnel to carry out conventional laboratory tests, their use in resource rich countries are less well defined. Indeed, the US Centers for Disease Control and Prevention and the American Association of Public Health Laboratories, in consultation with other sexually transmitted disease experts, concluded in 2017 that there was insufficient data to recommend the use of POCT for routine syphilis testing in the USFootnote 8.

By October 2, 2020, the US Food and Drug Administration licensed two syphilis POCT kitsFootnote 9: the Syphilis Health Check POC assay (Diagnostic Direct LLC, Stone Harbor, New Jersey or Trinity Biotech, Jamestown, New York) and the DPP POCT for HIV and Syphilis serology (Chembio Diagnostic System, Medford, New York). However, some evaluations of these test kits were based on small numbers of positive samples as well as serum specimens obtained by venipuncture rather than finger-prick whole blood, which is the specimen of choice for POCT (due to ease of obtaining specimen). Using finger-prick whole blood specimens, the Syphilis Health Check POCT had a sensitivity of 100% (with seven positive samples) when compared to consensus reference testing of RPR and treponemal enzyme immunoassay. This sensitivity decreased to 77.8% (with 18 positive samples) when compared with Syphilis Health Check POCT done with serum in the laboratory, and to only 50.0% sensitivity (with 16 positive samples) when compared with the treponemal enzyme immunoassayFootnote 10. Difficulty in reading the Syphilis Health Check POCT has also been reported for those specimens considered positive in the POCT but negative by consensus reference testing. A 2016 study that reported high sensitivity (94.7%) and specificity (100%) for the DPP POCT for HIV and Syphilis serology, the results for the syphilis component of the POCT were based on testing serum specimens and comparing the data to that obtained by the Treponema pallidum (T. pallidum) particle agglutination testFootnote 11. Currently, no syphilis POCT has been licensed in Canada.

In this study, our objective was to evaluate two syphilis POCTs (one has treponemal antigen and another that has both treponemal and non-treponemal antigens) to detect infectious syphilis and report on their preliminary performance based on studies done in the laboratory using serum samples with known syphilis serological test results.

Materials and methods

Patient serum samples and testing by conventional syphilis serology

One hundred serum samples from individual subjects, identified as either syphilis positive or syphilis negative at the Cadham Provincial Public Health Laboratory (Winnipeg, Manitoba), were provided in a blind fashion (regarding the syphilis test results) to the National Microbiology Laboratory for evaluation by POCT. Samples were defined as syphilis positive if tested positive by CMIA for antibodies to treponemal antigens. Samples were defined as syphilis negative when CMIA testing gave negative results. Qualitative RPR was performed on CMIA positive samples, followed by a quantitative VDRL test at the Cadham Provincial Public Health Laboratory. Both RPR and VDRL are considered as non-treponemal tests that detect non-treponemal antibodies, and positive results of either the RPR or VDRL tests (and in the presence of positive CMIA results) are indicative of active or infectious syphilis. Also, a different non-treponemal (VDRL) test was used to confirm the RPR qualitative results to increase the specificity of the non-treponemal test data.

Point-of-care test

Since there are no POCT kits licensed for sale in Canada, we obtained permission from Health Canada Special Access Program before the DPP® Syphilis Screen and Confirm Assay (ChemBio Diagnostic Systems, Inc., Medford, New York) and the RevealTM Rapid TP Antibody test (MedMira Laboratories, Inc., Halifax, Nova Scotia) were purchased directly from the suppliers. The Reveal Rapid TP Antibody POCT detects antibodies to synthetic peptides that resemble T. pallidum recombinant antigens and can be used with serum, whole blood or plasma samplesFootnote 12. The DPP Syphilis Screen and Confirm POCT detects antibodies to both treponemal and non-treponemal antigens in serum, whole blood or plasma specimensFootnote 13.

Two qualified laboratorians at the National Microbiology Laboratory carried out independent testing on each of the 100 serum samples with both POCT kits according to the manufacturers' instructions. Results were read manually and recorded as reactive, weakly reactive or very weakly reactive, even though any intensity of colour change in the test zone is considered reactive according to the package insert for both the Reveal Rapid TP Antibody POCT and the DPP Syphilis Screen and Confirm POCT. We chose to score the test results in more detail since reading intensity of colour change is subjective and difficulties in reading some syphilis POCTs have been described by others. Weakly reactive and very weakly reactive results were determined by comparison with the reactions given by the controls as well as other reactive samples obtained in the same run. When discordant results were obtained by the two independent operators, the test was repeated a second time by both operators and if the results matched on the second test, they were then used. In no circumstances, were any major discordant results encountered, such as reactive by one operator and non-reactive by another operator.

After the 100 samples have been tested by both operators using the two POCTs, results were provided to the investigators at the Cadham Provincial Public Health Laboratory, who then disclosed the CMIA, RPR and VDRL results to the investigators at the National Microbiology Laboratory to perform the analysis. The sensitivity (percent of infectious syphilis samples tested positive), specificity (percent of samples without infectious syphilis tested negative), false positive rate (percent of samples without infectious syphilis tested positive) and false negative rate (percent of infectious syphilis samples tested negative) of the POCTs for detection of infectious syphilis were calculated.

Results

Characteristics of clinical samples

Of the 100 samples provided for POCT evaluations, 80 were reactive or positive by CMIA (sample over cut-off, S/CO, values ranged from 1.11 to 27.11, mean=15.68, median=16.81) and 20 were non-reactive or negative (S/CO values ranged from 0.03 to 0.14, mean=0.61, median=0.05). The 20 negative CMIA samples did not undergo further testing and were regarded as syphilis negative for both active and past infections. The 80 CMIA positive samples were further divided into 40 that were non-reactive by RPR (which suggested absence of active syphilis infections but with treated past syphilis infections) and 40 that were RPR reactive (which suggested active syphilis infections). Among the 40 CMIA positive and RPR reactive samples, six were only weakly reactive by RPR. Of these six RPR weakly reactive samples, five were tested by VDRL to give reactive results with an undiluted sample. The remaining CMIA positive, RPR weakly reactive sample was VDRL reactive at 1:2 dilution. Of the 34 CMIA positive and RPR reactive samples, 14 had low or no VDRL titres (n=7 with VDRL titres of 1:2, n=6 with VDRL titres of 1:4, and n=1 was VDRL reactive at undiluted), while 20 samples were found to have VDRL titres of at least 1:8.

Performance of Reveal Rapid Treponema pallidum Antibody and DPP Syphilis Screen and Confirm tests

Table 1 shows the results of the Reveal Rapid TP Antibody POCT and the DPP Syphilis Screen and Confirm POCT on the three categories of serum samples (no syphilis infection, past syphilis infection and active syphilis infection). Sera from subjects with no evidence of current or past syphilis infection were found to be non-reactive by both POCTs. Only ten of the 40 samples categorised as having past syphilis infections were reactive in the Reveal Rapid TP Antibody POCT. The DPP Syphilis Screen and Confirm POCT detected antibodies to the treponemal antigen in 19 of the 40 samples from subjects with past syphilis infection; only one sample was reactive for both treponemal and non-treponemal antigens, while the remaining 20 were non-reactive for both antigens.

Table 1: Results of two syphilis point-of-care test kits showing number of reactive and non-reactive reactions on 100 serum specimens with known conventional syphilis serological findings
Syphilis statusTable 1 Footnote a Number of specimens Reveal Rapid T. pallidum Antibody point-of-care testTable 1 Footnote b DPP Syphilis Screen and Confirm point-of-care testTable 1 Footnote c
presented as treponemal/non-treponemal reactions
RTable 1 Footnote d (R)Table 1 Footnote d NRTable 1 Footnote e (NR)Table 1 Footnote e R/RTable 1 Footnote d (R/R)Table 1 Footnote d RTable 1 Footnote d/NR (R/NR)Table 1 Footnote dTable 1 Footnote e NRTable 1 Footnote e/RTable 1 Footnote d (NR/R)Table 1 Footnote dTable 1 Footnote e NRTable 1 Footnote e/NRTable 1 Footnote e (NR/NR)Table 1 Footnote e
No syphilis 20 0 0 20 20 0 0 0 0 0 0 20 20
Past syphilis 40 10 4 30 36 1 0 18 13 0 0 20Table 1 Footnote f 26Table 1 Footnote f
Active syphilisTable 1 Footnote g 40 38 36 2 4 35 27 5 8 0 0 0 5
With VDRL titer greater than 1:4 20 20 20 0 0 20 19 0 0 0 0 0 1
With VDRL titer 1:2 to 1:4 13 12 11 1 2 10 7 3 3 0 0 0 3
VDRL undiluted or RPR WRTable 1 Footnote h 7 6 5 1 2 5 1 2 5 0 0 0 1

The percentage of samples from those with active syphilis and reactive by both Reveal Rapid TP Antibody and DPP Syphilis Screen and Confirm POCTs are correlated to the quantitative non-treponemal test results, and this is particularly evident with the DPP Syphilis Screen and Confirm POCT (Table 1).

If very weakly reactive reactions were read conservatively as non-reactive (shown in parentheses in Tables 1 and Table 2), fewer samples from those with active and past syphilis infections were found to be reactive in both POCT. Twenty samples when tested by the Reveal Rapid TP Antibody POCT test gave either weakly reactive (n=12; three were past syphilis samples and nine were active syphilis samples) or very weakly reactive (n=8; six were past syphilis samples and two were active syphilis samples) reactions. In the DPP Syphilis Screen and Confirm POCT, 12 samples gave weakly reactive (n=1; past syphilis) or very weakly reactive (n=11; six past syphilis and five active syphilis) reactions in the treponemal antigen, while nine samples (n=8 active syphilis and one past syphilis) gave very weakly reactive results in the non-treponemal antigen (data not shown).

Table 2: Performance of Reveal Rapid TP Antibody point-of-care-test and DPP Syphilis Screen and Confirm point of care test to detect active or infectious syphilis based on laboratory evaluation of 100 serum samples defined by traditional syphilis serological assaysTable 2 Footnote a
Performance characteristics of syphilis point-of-care test Reveal Rapid T. pallidum AntibodyTable 2 Footnote b point-of-care testTable 2 Footnote c DPP Syphilis Screen and ConfirmTable 2 Footnote d point-of-care testTable 2 Footnote c
Overall sensitivityTable 2 Footnote e 95.0% (90.0%) 87.5% (67.5%)
SensitivityTable 2 Footnote f 100% (100%) 100% (95.0%)
SensitivityTable 2 Footnote g 92.3% (84.6%) 76.9% (53.8%)
SensitivityTable 2 Footnote h 85.7% (71.4%) 71.4% (14.3%)
Specificity 83.3% (93.3%) 98.3% (100%)
False positive rate 16.7% (6.7%) 1.7% (0%)
False negative rate 5.0% (10.0%) 12.5% (32.5%)

Table 2 compares the performance characteristics of the two syphilis POCTs for the 100 serum samples that were categorized as syphilis naive, past treated syphilis, active, or infectious syphilis. Sensitivity of both POCT for detection of infectious syphilis appeared to be affected by the quantitative non-treponemal test results of the samples. Both POCT gave 100% sensitivity in samples with VDRL titres of greater than 1:4, but sensitivity decreased stepwise in samples with lower VDRL titres (Table 2), and the DPP Syphilis Screen and Confirm POCT appeared to be affected more by samples with VDRL titres of 1:2 to 1:4 or lower. While the Reveal Rapid TP Antibody test has a higher sensitivity for detection of active syphilis, the DPP Syphilis Screen and Confirm assay has a better specificity.

Reproducibility of point-of-care test results between two independent operators

For the Reveal Rapid TP Antibody test, 91% of the results from both operators agreed while for the DPP Syphilis Screen and Confirm Assay, 94% and 95% of the results on the treponemal and non-treponemal components showed concordance between the two operators.

There were nine minor discrepancies between the two operators when testing the Reveal Rapid TP Antibody test, with operator #1 scoring nine samples as weakly reactive and operator #2 scoring seven as reactive and two as very weakly reactive. For the DPP Syphilis Screen and Confirm Assay, there were five discrepant results for the non-treponemal component, with operator #1 scoring them as reactive and operator #2 scoring them as very weakly reactive. For the treponemal component of the DPP Syphilis Screen and Confirm Assay, there were six discrepant results with operator #1 scoring them as reactive and operator #2 scoring them as very weakly reactive.

Discussion

This laboratory study indicated that both the Reveal Rapid TP Antibody and the DPP Syphilis Screen and Confirm POCTs had overall sensitivity and specificity of 85.0% or better, with the Reveal Rapid TP Antibody POCT showing an overall better sensitivity (95.0%) and DPP Syphilis Screen and Confirm showing better specificity (98.3%) for detection of infectious syphilis. Sera from subjects without current or past syphilis infection gave very clear-cut non-reactive results with both POCTs. Because the test samples were not random samples collected from the population, no attempt was made to extend the current results to calculate the positive and negative predictive values of these POCTs.

Most syphilis POCT kits on the market use treponemal antigen to detect anti-treponemal antibodies and only a handful of kits employ both treponemal and non-treponemal antigens that can simultaneously detect anti-treponemal as well as non-treponemal antibodies. Tests that detect anti-treponemal antibodies alone cannot be used to differentiate subjects with active or past infections because anti-treponemal antibodies tend to persist for a long time after the active infection disappears following successful treatment. Antibodies to both treponemal and non-treponemal antigens are indicative of active and infectious syphilis because antibodies to non-treponemal antigens usually show a gradual drop in titres and eventually disappear some months after the active infection is clear with treatment.

Although the Reveal Rapid TP Antibody POCT uses antigen(s) that represents T. pallidum proteins, and the DPP Syphilis Screen and Confirm POCT also has a treponemal antigen component, results suggested that both POCT favoured the detection of infectious syphilis rather than treated past syphilis. Antibodies to the treponemal antigen were detected in only 25.0% and 48.7% of serum samples from subjects with past syphilis when measured by the Reveal Rapid TP Antibody POCT and the DPP Syphilis Screen and Confirm POCT, respectively. In our study, when compared to results obtained by CMIA, the sensitivity of the treponemal component of the DPP Syphilis Screen and Confirm POCT to detect treponemal antibodies was 100% with serum samples from active syphilis cases but was only 48.7% with serum samples from past syphilis cases. The sensitivity of the Reveal Rapid TP Antibody POCT to detect treponemal antibodies was 95.0% in those with active syphilis but was only 25.0% in those with past syphilis.

This study shows the two POCT evaluated have good sensitivity and specificity for the detection of infectious syphilis. However, in those with low non-treponemal antibody titres, their performance may be compromised, thus confirming the findings by others for this category of active syphilis patientsFootnote 10Footnote 14Footnote 15. Of the two POCTs evaluated, the Reveal Rapid TP Antibody POCT may be affected less by the low non-treponemal antibody titres and was able to maintain reasonable sensitivity of 92.3% to 85.7% (to detect infectious syphilis) even in samples with VDRL antibody titres of 1:2 to 1:4 or only reactive in undiluted sera.

In the Reveal Rapid TP Antibody POCT, 25.0% of the CMIA positive samples were found to give either weakly or very weakly reactive results. In the DPP Syphilis Screen and Confirm POCT, 11.3% and 15.0% of the CMIA positive samples were found to give either weakly or very weakly reactive results to the non-treponemal and treponemal components, respectively. Both weakly reactive and especially very weakly reactive results may lead to difficulty in reading these tests in actual field use by non-trained operators. Also, the use of finger-prick whole blood specimens can potentially make reading the results even more difficult. Both training and clear guidelines on how to read POCT results may be required. Some POCT come with an electronic readerFootnote 11, and this may avoid inconsistencies in reading results between samples and operators. Another potential challenge in the implementation of POCT in sexually transmitted infection clinics or rural areas is related to the de-centralized syphilis testing and the resultant difficulties in the capturing of test results by public health for surveillance purpose and for the subsequent development of public health intervention policies.

One of the reasons that has been put forward to explain the recent increase in syphilis infection in Canada (in both urban and remote or rural locations) is the inequity of accessibility within the healthcare system, including the testing facilities for infectionsFootnote 5Footnote 6. Vulnerable populations, whether living in urban or rural areas, can be hard to reach due to homeless or unstable housing, mistrust in the healthcare system or lack of accessible testing facilities in remote localities. Alternate testing approaches, such as the use of POCT, may help to meet the need created by the circumstances of these at-risk populations. Current POCT provide only qualitative results of positive or negative findings, and the lack of quantitative result on antibody titres may prevent their use for monitoring response to treatment as well as differentiating repeated or reinfections from past infections. This contrasts with the current practice of using a quantitative non-treponemal test result (such as RPR or VDRL titres) to monitor for either a decrease in antibody titres as proof of positive response to treatment or a rising antibody titer in the case of no response to treatment or an occurrence of reinfection.

Limitations

There are several limitations in the interpretation of this study. First, serum samples collected for conventional syphilis serology were used instead of finger prick whole blood specimens, which will likely be the sample of choice when these tests are used in the field. Secondly, these assays were done in a controlled laboratory environment by trained laboratory staff with experience in carrying out clinical diagnostic tests; therefore, actual performance of these POCT in the field or in a real-world situation may differ. Another limitation is the small number of samples tested, especially in the subcategories of active syphilis cases with different VDRL titres and, as such, the sensitivity data may be inaccurate. Finally, the usefulness or performance of a POCT would also depend on the setting or prevalence of the disease where the POCT will be deployed.

Conclusion

The two POCT evaluated in this laboratory study appeared to show promising results for detection of infectious syphilis especially in those with non-treponemal antibody titres equal to or greater than 1:4, but not for detection of past syphilis. Further evaluations in the field will be required in order to confirm the findings in this preliminary study. Field evaluations and clinical studies will offer further experience in the use of syphilis POCT that may ultimately contribute towards better control of infectious and congenital syphilis. Data from this kind of study may also be useful for potential future licensure of such test kits in Canada.

Authors' statement

RSWT — Designed the study, analysed the results, prepared the first draft

DS — Designed the study

MS — Carried the test and analysed the results

KH — Carried the test and analysed the results

All authors commented and approved the manuscript for submission.

Results and opinion stated in this manuscript are those of the authors and they do not necessarily represent the position of neither the National Microbiology Laboratory nor the Public Health Agency of Canada.

Competing interests

None.

Acknowledgements

The authors thanked ML Lam for assistance in preparation of the manuscript and A Gretchen for preparation of the serology panel.

Funding

This study did not receive any external funding and was supported by funding from the Public Health Agency of Canada.

Page details

Date modified: