Evaluation of syphilis reactor grids: optimizing impact

An Automated Syphilis Serology Record Search and Review Algorithm to Prioritize Investigations by Health Departments

BACKGROUND

Reactive syphilis serologies are investigated by health departments to determine if they represent new infection, reinfection, or treatment failure. Serologies prioritized for investigation based on nontreponemal test titer and age (using a "reactor grid") undergo manual record search and review. We developed a computerized algorithm that automates the record search and review.

METHODS

We developed and tested the algorithm using a Florida Department of Health data set containing serologies reported January 2016 to December 2018 and previous records linked to each individual. The algorithm was based on the syphilis case definition, which requires (except primary cases with signs and symptoms) (1) a positive treponemal test result and a newly positive nontreponemal test result or (2) a 4-fold increase in nontreponemal test titer. Two additional steps were added to avoid missing cases. New York City Department of Health and Mental Hygiene validated this algorithm.

RESULTS

The algorithm closed more investigations (49.9%) than the reactor grid (27.0%). The algorithm opened 99.4% of the individuals investigated and labeled as cases by the health department; it missed 75 cases. Many investigations opened by the algorithm were closed by the reactor grid; we could not assess how many would have been cases. In New York City, the algorithm closed 70.9% of investigations, likely because more individuals had previous test in the database (88.2%) compared with Florida (56.5%).

CONCLUSIONS

The automated algorithm successfully searched and reviewed records to help identify cases of syphilis. We estimate the algorithm would have saved Florida 590 workdays for 3 years.

Reactor Grids for Prioritizing Syphilis Investigations: Are Primary Syphilis Cases Being Missed?

BACKGROUND

Health departments prioritize investigations of reported reactive serologic tests based on age, gender, and titer using reactor grids. We wondered how reactor grids are used in different programs, and if administratively closing investigations of low-titer tests could lead to missed primary syphilis cases.

METHODS

We obtained a convenience sample of reactor grids from 13 health departments. Interviews with staff from several jurisdictions described the role of grids in surveillance and intervention. From 5 jurisdictions, trends in reactive nontreponemal tests and syphilis cases over time (2006-2015) were assessed by gender, age, and titer. In addition, nationally-reported primary syphilis cases (2013-2015) were analyzed to determine what proportion had low titers (≤1:4) that might be administratively closed by grids without further investigation.

RESULTS

Grids and follow-up approaches varied widely. Health departments in the study received a total of 48,573 to 496,503 reactive serologies over a 10-year period (3044-57,242 per year). In 2006 to 2015, the number of reactive serologies increased 37% to 169%. Increases were largely driven by tests for men although the ratios of tests per reported case remained stable over time. Almost one quarter of reported primary syphilis had low titers that would be excluded by most grids. The number of potentially missed primary syphilis cases varied by gender and age with 41- to 54-year-old men accounting for most.

CONCLUSIONS

Reactor grids that close tests with low titers or from older individuals may miss some primary syphilis cases. Automatic, computerized record searches of all reactive serologic tests could help improve prioritization.

Going Off Grid: Modeling an Automated Record Search to Process Electronically Reported Reactive Nontreponemal Syphilis Tests

BACKGROUND

Before searching prior records, sexually transmitted disease programs use syphilis reactor grids to exclude some reactive nontreponemal test results (RNTs) based on patient age, gender, and test titer. We propose a new algorithm that starts with comparing RNTs to previous syphilis nontreponemal tests and current treponemal test results.

METHODS

Deduplicated RNTs from Florida's surveillance system (2006-2015) were extracted and stratified on morbidity. An algorithm was developed to triage RNTs. Sensitivity and specificity of the algorithm and the current reactor grid were estimated using reported syphilis cases. A random sample of cases missed by the proposed algorithm, stratified by stage of disease, was reviewed to verify case classification.

RESULTS

Reported RNTs increased 58% from 2006 (n = 34,808) to 2015 (n = 55,001) (total = 372,902). The current reactor grid removed 91,518 (24.5%) RNTs and missed 1149 potential cases. Strictly following the reactor grid would result in a sensitivity of 97.4% and a specificity of 27.5%. The proposed algorithm would remove 242,078 (64.9%) RNTs and miss 2768 potential cases. This results in a slightly lower sensitivity of 93.8%, but nearly triples the specificity, 72.9%. A review of a random sample of the 2768 cases estimated that 72.7% would not have met the syphilis surveillance case definition, resulting in an adjusted sensitivity of 98.4%.

CONCLUSIONS

In Florida, an algorithm that starts by searching previous syphilis test results vastly improved specificity and slightly improved sensitivity compared with the current reactor grid. Implementing an automated algorithm could increase case ascertainment efficiency and further prioritize likely cases for investigation.

A Web Application to Facilitate Syphilis Reactor Grid Evaluations

BACKGROUND

Many health departments use a "reactor grid" to determine which laboratory-reported syphilis serologic test results require investigation. We developed a Web-based tool, the Syphilis Reactor Grid Evaluator (SRGE), to facilitate health department reactor grid evaluations and test the tool using data from Seattle & King County, Washington.

METHODS

We developed SRGE using the R Shiny Web application framework. When populated with a data set including titer results and final disposition codes, SRGE displays the percent of verified early syphilis cases by serologic titer result and patient age in each cell of the grid. The results can be optionally stratified by sex, test type, and previous rapid plasma reagin titer. The impact of closing laboratory results without investigation in cells selected by the user is dynamically computed. The SRGE calculates the percent of all laboratory reports closed ("efficiency gained"), the proportion of all early syphilis cases closed without investigation ("case finding loss"), and the ratio of percent of cases identified for investigation to percent of all laboratory reports investigated ("efficiency ratio"). After defining algorithms, users can compare them side-by-side, combine subgroup-specific algorithms, and export results. We used SRGE to compare the current Public Health-Seattle & King County (PHSKC) reactor grid to 5 alternate algorithms.

RESULTS

Of 13,504 rapid plasma reagin results reported to PHSKC from January 1, 2006, to December 31, 2015, 1565 were linked to verified early syphilis cases. Updating PHSKC's current reactor grid could result in an efficiency gain of 4.8% to 25.2% (653-3403 laboratory reports) and case finding loss of 1% to 8.4% (10-99 fewer cases investigated).

CONCLUSIONS

The Syphilis Reactor Grid Evaluator can be used to rapidly evaluate alternative approaches to optimizing the reactor grid. Changing the reactor grid in King County to close more laboratory results without investigation could improve efficiency with minimal impact on syphilis case finding.

Towards probabilistic decision support in public health practice: predicting recent transmission of tuberculosis from patient attributes

OBJECTIVE

Investigating the contacts of a newly diagnosed tuberculosis (TB) case to prevent TB transmission is a core public health activity. In the context of limited resources, it is often necessary to prioritize investigation when multiple cases are reported. Public health personnel currently prioritize contact investigation intuitively based on past experience. Decision-support software using patient attributes to predict the probability of a TB case being involved in recent transmission could aid in this prioritization, but a prediction model is needed to drive such software.

METHODS

We developed a logistic regression model using the clinical and demographic information of TB cases reported to Montreal Public Health between 1997 and 2007. The reference standard for transmission was DNA fingerprint analysis. We measured the predictive performance, in terms of sensitivity, specificity, negative predictive value, positive predictive value, the Receiver Operating Characteristic (ROC) curve and the Area Under the ROC (AUC).

RESULTS

Among 1552 TB cases enrolled in the study, 314 (20.2%) were involved in recent transmission. The AUC of the model was 0.65 (95% confidence interval: 0.61-0.68), which is significantly better than random prediction. The maximized values of sensitivity and specificity on the ROC were 0.53 and 0.67, respectively.

CONCLUSIONS

The characteristics of a TB patient reported to public health can be used to predict whether the newly diagnosed case is associated with recent transmission as opposed to reactivation of latent infection.

The footprint of old syphilis: using a reverse screening algorithm for syphilis testing in a U.S. Geographic Information Systems-Based Community Outreach Program

The impact of syphilis reverse sequence screening has not been evaluated in community outreach. Using reverse sequence screening in neighborhoods identified with geographic information systems, we found that among 239 participants, 45 (19%) were seropositive. Of these, 3 (7%) had untreated syphilis, 33 (73%) had previously treated syphilis infection, and 9 (20%) had negative nontreponemal test results.