Verification and classification bias interactions in diagnostic test accuracy studies for fine-needle aspiration biopsy

Real-World Diagnostic Accuracy of the On-Site Cytopathology Advance Report (OSCAR) Procedure Performed in a Multidisciplinary One-Stop Breast Clinic

Fine-needle aspiration (FNA) cytology has been widely used for the diagnosis of breast cancer lesions with the objective of differentiating benign from malignant masses. However, the occurrence of unsatisfactory samples and false-negative rates remains a matter of concern. Major improvements have been made thanks to the implementation of rapid on-site evaluation (ROSE) in multidisciplinary and integrated medical settings such as one-stop clinics (OSCs). In these settings, clinical and radiological examinations are combined with a morphological study performed by interventional pathologists. The aim of our study was to assess the diagnostic accuracy of the on-site cytopathology advance report (OSCAR) procedure on breast FNA cytologic samples in our breast OSC during the first three years (April 2004 till March 2007) of its implementation. To this goal, we retrospectively analyzed a series of 1820 breast masses (1740 patients) radiologically classified according to the American College of Radiology (ACR) BI-RADS lexicon (67.6% being either BI-RADS 4 or 5), sampled by FNA and immediately diagnosed by cytomorphology. The clinicoradiological, cytomorphological, and histological characteristics of all consecutive patients were retrieved from the hospital computerized medical records prospectively registered in the central information system. Histopathological analysis and ultrasound (US) follow-up (FU) were the reference diagnostic tests of the study design. In brief, we carried out either a histopathological verification or an 18-month US evaluation when a benign cytology was concordant with the components of the triple test. Overall, histology was available for 1138 masses, whereas 491 masses were analyzed at the 18-month US-FU. FNA specimens were morphologically nondiagnostic in 3.1%, false negatives were observed in 1.5%, and there was only one false positive (0.06%). The breast cancer prevalence was 62%. Diagnostic accuracy measures of the OSCAR procedure with their 95% confidence intervals (95% CI) were the following: sensitivity (Se) = 97.4% (96.19-98.31); specificity (Sp) = 94.98% (92.94-96.56); positive predictive value (PPV) = 96.80% (95.48-97.81); negative predictive value (NPV) = 95.91% (94.02-97.33); positive likelihood ratio (LR+) = 19.39 (13.75-27.32); negative predictive ratio (LR-) = 0.03 (0.02-0.04), and; accuracy = 96.45% (95.42-97.31). The respective positive likelihood ratio (LR+) for each of the four categories of cytopathological diagnoses (with their 95% CI) which are malignant, suspicious, benign, and nondiagnostic were 540 (76-3827); 2.69 (1.8-3.96); 0.03 (0.02-0.04); and 0.37 (0.2-0.66), respectively. In conclusion, our study demonstrates that the OSCAR procedure is a highly reliable diagnostic approach and a perfect test to select patients requiring core-needle biopsy (CNB) when performed by interventional cytopathologists in a multidisciplinary and integrated OSC setting. Besides drastically limiting the rate of nondiagnostic specimens and diagnostic turn-around time, OSCAR is an efficient and powerful first-line diagnostic approach for patient-centered care.

Partial Verification Bias Correction Using Inverse Probability Bootstrap Sampling for Binary Diagnostic Tests

In medical care, it is important to evaluate any new diagnostic test in the form of diagnostic accuracy studies. These new tests are compared to gold standard tests, where the performance of binary diagnostic tests is usually measured by sensitivity (Sn) and specificity (Sp). However, these accuracy measures are often biased owing to selective verification of the patients, known as partial verification bias (PVB). Inverse probability bootstrap (IPB) sampling is a general method to correct sampling bias in model-based analysis and produces debiased data for analysis. However, its utility in PVB correction has not been investigated before. The objective of this study was to investigate IPB in the context of PVB correction under the missing-at-random assumption for binary diagnostic tests. IPB was adapted for PVB correction, and tested and compared with existing methods using simulated and clinical data sets. The results indicated that IPB is accurate for Sn and Sp estimation as it showed low bias. However, IPB was less precise than existing methods as indicated by the higher standard error (SE). Despite this issue, it is recommended to use IPB when subsequent analysis with full data analytic methods is expected. Further studies must be conducted to reduce the SE.

Community-level prevalence of epilepsy and of neurocysticercosis among people with epilepsy in the Balaka district of Malawi: A cross-sectional study

Background

Epilepsy and neurocysticercosis (NCC) prevalence estimates in sub-Saharan Africa are still scarce but show important variation due to the population studied and different screening and diagnosis strategies used. The aims of this study were to estimate the prevalence of epileptic seizures and epilepsy in the sampled population, and the proportion of NCC among people with epilepsy (PWE) in a large cross-sectional study in a rural district of southern Malawi.

Methods

We conducted a community-based door-to-door screening study for epileptic seizures in Balaka, Malawi between October and December 2012. Past epileptic seizures were reported through a 15-item questionnaire answered by at least one person per household generating five major criteria. People who screened positive were further examined by a neurologist to establish diagnosis. Patients diagnosed with epilepsy were examined and offered Taenia solium cyst antigen and antibody serological tests, and a CT scan for the diagnosis of NCC.

Results

In total, screening information on 69,595 individuals was obtained for lifetime occurrence of epileptic seizures. 3,100 (4.5%) participants screened positive, of whom 1,913 (62%) could be followed-up and underwent further assessment. Lifetime prevalence was 3.0% (95% Bayesian credible interval [CI] 2.8 to 3.1%) and 1.2% (95%BCI 0.9 to 1.6%) for epileptic seizures and epilepsy, respectively. NCC prevalence among PWE was estimated to be 4.4% (95%BCI 0.8 to 8.5%). A diagnosis of epilepsy was ultimately reached for 455 participants.

Conclusion

The results of this large community-based study contribute to the evaluation and understanding of the burden of epilepsy in the population and of NCC among PWE in sub-Saharan Africa.

Correcting for partial verification bias in diagnostic accuracy studies: A tutorial using R

Diagnostic tests play a crucial role in medical care. Thus any new diagnostic tests must undergo a thorough evaluation. New diagnostic tests are evaluated in comparison with the respective gold standard tests. The performance of binary diagnostic tests is quantified by accuracy measures, with sensitivity and specificity being the most important measures. In any diagnostic accuracy study, the estimates of these measures are often biased owing to selective verification of the patients, which is referred to as partial verification bias. Several methods for correcting partial verification bias are available depending on the scale of the index test, target outcome, and missing data mechanism. However, these are not easily accessible to the researchers due to the complexity of the methods. This article aims to provide a brief overview of the methods available to correct for partial verification bias involving a binary diagnostic test and provide a practical tutorial on how to implement the methods using the statistical programming language R.

The impact of imperfect screening tools on measuring the prevalence of epilepsy and headaches in Burkina Faso

BACKGROUND

Epilepsy and progressively worsening severe chronic headaches (WSCH) are the two most common clinical manifestations of neurocysticercosis, a form of cysticercosis. Most community-based studies in sub-Saharan Africa (SSA) use a two-step approach (questionnaire and confirmation) to estimate the prevalence of these neurological disorders and neurocysticercosis. Few validate the questionnaire in the field or account for the imperfect nature of the screening questionnaire and the fact that only those who screen positive have the opportunity to be confirmed. This study aims to obtain community-based validity estimates of a screening questionnaire, and to assess the impact of verification bias and misclassification error on prevalence estimates of epilepsy and WSCH.

METHODOLOGY/PRINCIPAL FINDINGS

Baseline screening questionnaire followed by neurological examination data from a cluster randomized controlled trial collected between February 2011 and January 2012 were used. Bayesian latent-class models were applied to obtain verification bias adjusted validity estimates for the screening questionnaire. These models were also used to compare the adjusted prevalence estimates of epilepsy and WSCH to those directly obtained from the data (i.e. unadjusted prevalence estimates). Different priors were used and their corresponding posterior inference was compared for both WSCH and epilepsy. Screening data were available for 4768 individuals. For epilepsy, posterior estimates for the sensitivity varied with the priors used but remained robust for the specificity, with the highest estimates at 66.1% (95%BCI: 56.4%;75.3%) for sensitivity and 88.9% (88.0%;89.8%) for specificity. For WSCH, the sensitivity and specificity estimates remained robust, with the highest at 59.6% (49.7%;69.1%) and 88.6% (87.6%;89.6%), respectively. The unadjusted prevalence estimates were consistently lower than the adjusted prevalence estimates for both epilepsy and WSCH.

CONCLUSIONS/SIGNIFICANCE

This study demonstrates that in some settings, the prevalence of epilepsy and WSCH can be considerably underestimated when using the two-step approach. We provide an analytic solution to obtain more valid prevalence estimates of these neurological disorders, although more community-based validity studies are needed to reduce the uncertainty of the estimates. Valid estimates of these two neurological disorders are essential to obtain accurate burden values for neglected tropical diseases such as neurocysticercosis that manifest as epilepsy or WSCH.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03095339.