Modeling conditional dependence among multiple diagnostic tests

Diagnostic accuracy of an immunomagnetic separation-PCR assay to detect pathogenic Leptospira spp. in urine from dairy cattle, using a Bayesian latent class model

Leptospirosis is a zoonotic disease that has spread worldwide and causes significant economic losses in the dairy industry. The causal agents of this infectious disease are members of the genus Leptospira, known as pathogenic Leptospira spp. Specific clinical signs of the infection are difficult to detect. Therefore, the disease is normally under-diagnosed, mostly due to the lack of a cost-effective technique for diagnosing animals with a low bacterial load in their urine. The aim of this study was to assess the diagnostic accuracy of a qPCR coupled with a previous Immunomagnetic separation (IMS) step (IMS-qPCR) against a qPCR without using IMS, using a Bayesian latent class model (2 tests, 3 populations) to determine the leptospirosis infectious status in naturally infected dairy cattle. The results revealed that IMS qPCR had a sensitivity (Se) of 95.7% (95% Probability Interval (PI) = 85.0; 99.4%) and a specificity (Sp) of 98% (95% PI = 96.1; 99.4%), indicating that it is more sensitive than conventional qPCR (Se = 69.7% (95% PI = 59.2; 79.0%); median difference = 25.2% (Monte Carlo Error = 10.2%); and the Sp = 98.8% (95% PI = 97.6; 99.5%), median difference = 0.8% (Monte Carlo Error = 2.1%). Therefore, results shows that IMS-qPCR is a more useful diagnostic tool in terms of accuracy for detecting infectious animals with pathogenic Leptospira in their urine.

Estimating sensitivity and specificity of diagnostic tests using latent class models that account for conditional dependence between tests: a simulation study

BACKGROUND

Latent class models are increasingly used to estimate the sensitivity and specificity of diagnostic tests in the absence of a gold standard, and are commonly fitted using Bayesian methods. These models allow us to account for 'conditional dependence' between two or more diagnostic tests, meaning that the results from tests are correlated even after conditioning on the person's true disease status. The challenge is that it is not always clear to researchers whether conditional dependence exists between tests and whether it exists in all or just some latent classes. Despite the increasingly widespread use of latent class models to estimate diagnostic test accuracy, the impact of the conditional dependence structure chosen on the estimates of sensitivity and specificity remains poorly investigated.

METHODS

A simulation study and a reanalysis of a published case study are used to highlight the impact of the conditional dependence structure chosen on estimates of sensitivity and specificity. We describe and implement three latent class random-effect models with differing conditional dependence structures, as well as a conditional independence model and a model that assumes perfect test accuracy. We assess the bias and coverage of each model in estimating sensitivity and specificity across different data generating mechanisms.

RESULTS

The findings highlight that assuming conditional independence between tests within a latent class, where conditional dependence exists, results in biased estimates of sensitivity and specificity and poor coverage. The simulations also reiterate the substantial bias in estimates of sensitivity and specificity when incorrectly assuming a reference test is perfect. The motivating example of tests for Melioidosis highlights these biases in practice with important differences found in estimated test accuracy under different model choices.

CONCLUSIONS

We have illustrated that misspecification of the conditional dependence structure leads to biased estimates of sensitivity and specificity when there is a correlation between tests. Due to the minimal loss in precision seen by using a more general model, we recommend accounting for conditional dependence even if researchers are unsure of its presence or it is only expected at minimal levels.

Evaluation of tuberculosis diagnostic test accuracy using Bayesian latent class analysis in the presence of conditional dependence between the diagnostic tests used in a community-based tuberculosis screening study

Diagnostic accuracy studies in pulmonary tuberculosis (PTB) are complicated by the lack of a perfect reference standard. This limitation can be handled using latent class analysis (LCA), assuming independence between diagnostic test results conditional on the true unobserved PTB status. Test results could remain dependent, however, e.g. with diagnostic tests based on a similar biological basis. If ignored, this gives misleading inferences. Our secondary analysis of data collected during the first year (May 2018 -May 2019) of a community-based multi-morbidity screening program conducted in the rural uMkhanyakude district of KwaZulu Natal, South Africa, used Bayesian LCA. Residents of the catchment area, aged ≥15 years and eligible for microbiological testing, were analyzed. Probit regression methods for dependent binary data sequentially regressed each binary test outcome on other observed test results, measured covariates and the true unobserved PTB status. Unknown model parameters were assigned Gaussian priors to evaluate overall PTB prevalence and diagnostic accuracy of 6 tests used to screen for PTB: any TB symptom, radiologist conclusion, Computer Aided Detection for TB version 5 (CAD4TBv5≥53), CAD4TBv6≥53, Xpert Ultra (excluding trace) and culture. Before the application of our proposed model, we evaluated its performance using a previously published childhood pulmonary TB (CPTB) dataset. Standard LCA assuming conditional independence yielded an unrealistic prevalence estimate of 18.6% which was not resolved by accounting for conditional dependence among the true PTB cases only. Allowing, also, for conditional dependence among the true non-PTB cases produced a 1.1% plausible prevalence. After incorporating age, sex, and HIV status in the analysis, we obtained 0.9% (95% CrI: 0.6, 1.3) overall prevalence. Males had higher PTB prevalence compared to females (1.2% vs. 0.8%). Similarly, HIV+ had a higher PTB prevalence compared to HIV- (1.3% vs. 0.8%). The overall sensitivity for Xpert Ultra (excluding trace) and culture were 62.2% (95% CrI: 48.7, 74.4) and 75.9% (95% CrI: 61.9, 89.2), respectively. Any chest X-ray abnormality, CAD4TBv5≥53 and CAD4TBv6≥53 had similar overall sensitivity. Up to 73.3% (95% CrI: 61.4, 83.4) of all true PTB cases did not report TB symptoms. Our flexible modelling approach yields plausible, easy-to-interpret estimates of sensitivity, specificity and PTB prevalence under more realistic assumptions. Failure to fully account for diagnostic test dependence can yield misleading inferences.

Bayesian Evaluation of Three Serological Tests for Diagnosis of Brucella infections in Dromedary Camels Using Latent Class Models

Brucellosis is a zoonotic disease with significant economic and public health impacts. The disease has been found in ruminants, including camels, but clinical diagnosis of camel brucellosis is difficult due to the lack of clinical signs. Thus, this study aimed to estimate the sensitivity (Se) and specificity (Sp) of the Buffered Plate Antigen Test (BPAT), Rose Bengal Test (RBT), and indirect ELISA (i-ELISA) for the diagnosis of Brucella infection in dromedary camels imported from Sudan to Egypt. The secondary objective of the study was to calculate the animal-level true prevalence of Brucella infection in imported camels. A cross-sectional study was carried out on 921 apparently healthy camels randomly selected from those imported from Sudan and kept in the quarantine stations in the Shalateen area of the Red Sea Governorate, Egypt, between June 2018 and January 2019. Serum samples were collected and analyzed using BPAT, RBT, and i-ELISA. The posterior estimates [medians and 95% Bayesian probability intervals (95% BPI)] for Se and Sp of the three serological tests were obtained using Bayesian latent class models (BLCMs). The BLCM was fitted with the assumption that the BPAT and RBT tests were conditionally dependent on the true brucellosis status of camels. All tests had comparable and high Se (>86%) and Sp (>98%). The animal-level true prevalence of Brucella infection in imported camels was 8.6% (95% BPI: 6.8 - 10.7). Based on these findings, the three assays could be used for the initial screening of Brucella infection in camels. However, the BPAT and RBT are more suitable for use in camel brucellosis control and eradication program in Egypt because of their low unit cost and fast turnaround time compared to the i-ELISA. In addition, BPAT and RBT could be performed in the field where in-vivo tests are rarely used due to logistic and management constraints.

Meta-analysis of dichotomous and ordinal tests with an imperfect gold standard

Standard methods for the meta-analysis of medical tests, without assuming a gold standard, are limited to dichotomous data. Multivariate probit models are used to analyse correlated dichotomous data, and can be extended to model ordinal data. Within the context of an imperfect gold standard, they have previously been used for the analysis of dichotomous and ordinal test data from a single study, and for the meta-analysis of dichotomous tests. However, they have not previously been used for the meta-analysis of ordinal tests. In this article, we developed a Bayesian multivariate probit latent class model for the simultaneous meta-analysis of ordinal and dichotomous tests without assuming a gold standard, which also allows one to obtain summary estimates of joint test accuracy. We fitted the models using the software Stan, which uses a state-of-the-art Hamiltonian Monte Carlo algorithm, and we applied the models to a dataset in which studies evaluated the accuracy of tests, and test combinations, for deep vein thrombosis. We demonstrate the issues with dichotomising ordinal test accuracy data in the presence of an imperfect gold standard, before applying and comparing several variations of our proposed model which do not require the data to be dichotomised. The models proposed will allow researchers to more appropriately meta-analyse ordinal and dichotomous tests without a gold standard, potentially leading to less biased estimates of test accuracy. This may lead to a better understanding of which tests, and test combinations, should be used for any given medical condition.

Bayesian estimation of sensitivity and specificity of fecal culture, fecal PCR and serum ELISA for diagnosis of Mycobacterium avium subsp. paratuberculosis infections in sheep

The objective of the present study was to evaluate the diagnostic accuracy of the individual fecal culture (IFC), fecal PCR (FPCR), and serum ELISA for the detection of Mycobacterium avium subsp. paratuberculosis (MAP) infections in sheep from four governorates in Egypt, using a latent class model (LCM) fitted within a Bayesian framework. Furthermore, the within-governorate prevalence of MAP infection in sheep was estimated as a secondary objective. Fecal and blood samples were collected from 370 sheep in four Egyptian governorates. Fecal samples were analyzed by IFC and RT-PCR based on ISMav2 gene, while ELISA was performed on serum samples. The sensitivity (Se) and specificity (Sp) of the three diagnostic tests were estimated using a three-tests-four-populations Bayesian LCM to obtain posterior estimates [medians and 95% Bayesian credible intervals (95% BCI)] for each parameter. The median Se estimates (95% BCI) for IFC, FPCR, and serum ELISA were 31.8% (22.8-41.4), 49.7% (31.8-79.9), and 61.2% (39.8-81.4), respectively. The median Sp estimates (95% BCI) for IFC, FPCR, and serum ELISA were 97.7% (96.1-98.9), 97.7% (95.6-99.5), and 98.4% (96.9-99.3), respectively. The median within-governorate paratuberculosis prevalence (95% BCI) was 5.2% (1.1-13.6), 8.4% (2.9-17.7), 9.4% (3.0-20.7), and 18.2% (10.5-29.5) for the Gharbia, Menoufia, Qalyubia, and Kafr El-Sheikh governorates, respectively. In conclusion, at a ratio of the optical density (OD) sample/OD positive control threshold of > 45%, ELISA showed the highest Se among the three tests and comparable Sp to IFC and FPCR. The test ELISA evaluated in this study is an interesting alternative for detecting MAP in sheep due to its higher Se, lower cost, and shorter turnaround laboratory time compared to IFC and FPCR.

Examining the correlation between treatment effects in clinical trials and economic modelling

INTRODUCTION

Many diseases have a sequential treatment pathway. Compared with patients without previous treatment, patients who fail initial treatment may have lower success rates with a second treatment. This phenomenon can be explained by a correlation between treatment effects.

METHODS

We developed a statistical model of covariance for the underlying unobserved correlation between treatments and established a mathematical expression for the magnitude of the latent correlation term. We conducted a simulation study of clinical trials to investigate the correlation between two treatments and explored clinical examples based on published literature to illustrate the identification and evaluation of these correlations.

RESULTS

Our simulation study confirmed that a treatment correlation reduces the probability of success for the second treatment, compared with no correlation. We found that treatment correlations may be observable in clinical trials, such as for depression and lung cancer, and the magnitude of correlation may be estimated. We illustrated that treatment correlations can be incorporated into an economic model, with possible impacts on cost-effectiveness results. Additional applications of correlation concepts are also discussed.

CONCLUSIONS

We evaluated the correlation between treatment effects and our approach can be applied to clinical trial design and economic modelling of sequential clinical treatment pathways.

Reproductive tract disease in Irish grazing dairy cows: Retrospective observational study examining its association with reproductive performance and accuracy of 2 diagnostic tests

The detection of reproductive tract disease (RTD) 3 wk postpartum is important because of its effect on subsequent reproductive outcomes. Numerous methods for the diagnosis of RTD are described, some of which are more practical and instantaneous in terms of diagnosis. Two of these methods involve identification of purulent vaginal discharge (PVD) and evidence of ultrasonographic uterine changes indicative of endometritis (UE). The objectives of our retrospective observational study were (1) to assess the association of PVD or UE score at the prebreeding examination (PBE) with the hazard of pregnancy within the subsequent breeding season; (2) to determine the test sensitivity (Se) and specificity (Sp) at the point of sampling of both tests using a Bayesian latent class model; and (3) to determine the effect of varying positivity thresholds on test accuracy. To achieve these objectives, we analyzed an initial data set of 5,049 PBE from 2,460 spring-calved cows in 8 herds between 2014 and 2018. Each PBE was conducted once between 25 and 86 d in milk. At each PBE, vaginal discharge was obtained with a Metricheck device (Simcro) whereas uterine contents were assessed using transrectal ultrasonography. Purulent vaginal discharge was scored on a scale of 0 to 3 depending on discharge character, and UE was scored on a scale of 0 to 4 depending on the presence and consistency of intraluminal fluid. Cows with scores of ≥2 in either test had received treatment. Fertility data were available from 4,756 PBE after data exclusion. The association between PVD or UE score at the PBE and subsequent hazard of pregnancy was analyzed using a Cox proportional hazards model. Cows with a PVD score of 2 or 3 were less likely to conceive than cows with a PVD score 0 [score 2 hazard ratio (HR) = 0.74; 95% confidence interval (CI): 0.59-0.94; score 3 HR = 0.65; 95% CI: 0.51-0.84]. Cows with a UE score of 1, 2, 3, or 4 were less likely to conceive than cows with a UE score of 0 (score 1 HR = 0.82; 95% CI: 0.73-0.93; score 2 HR = 0.79; 95% CI: 0.62-1.00; score 3 HR = 0.43; 95% CI: 0.43-0.90; score 4 HR = 0.39; 95% CI: 0.26-0.58). To determine the Se and Sp of PVD or UE score for diagnosis of RTD at the time of PBE, a Bayesian latent class model was fitted on 2,460 individual cow PBE. Flat priors were used for the Se and Sp of UE, whereas informative priors were used for PVD Se (mode = 65%, 5th percentile = 45%) and Sp (mode = 90%, 5th percentile = 80%) and RTD prevalence (mode = 20%, 5th percentile = 10%). Posterior estimates (median and 95% Bayesian probability intervals; BPI) were obtained using 'rjags' (R Studio). The optimal test thresholds (PVD and UE score ≥1) were selected by assessing the effect of different thresholds on test estimates and using a misclassification cost analysis. Based on these, median (95% BPI) Se for PVD and UE score ≥1 were 44% (29-60%) and 67% (33-100%), respectively. Median Sp for PVD and UE score ≥1 were 90% (86-93%) and 91% (86-93%), respectively. Higher scores in both tests were associated with impaired fertility, and UE scoring with a threshold of ≥1 had the highest test Se and Sp estimates although test Se was conditional on days in milk when the PBE occurred.

Developing Economic Models for Assessing the Cost-Effectiveness of Multiple Diagnostic Tests: Methods and Applications

Background

Clinical pathways with multiple diagnostic tests are complex to model, but problematic and simplistic approaches are often used in economic evaluations.

Methods

We analyzed statistical methods of handling multiple diagnostic tests and provided guidance on applying these methods in economic modeling. We first introduced a statistical model to quantify the correlations between 2 tests and how those correlations can be incorporated within an economic model. We also presented the general form of conditional dependence among multiple tests. We then introduced net reclassification improvement (NRI), a measure that evaluates the added value of a new risk factor (e.g., biomarker) for risk prediction. We further provided 2 examples to illustrate the application of these methods.

Results

Our first example illustrated how to model an add-on test to an existing test, in the absence of a perfect reference standard. After accounting for the imperfect nature of both tests and the conditional dependence between tests, the potential health benefits from the additional test were reduced. This led to differential cost-effectiveness results when comparing models using the perfect test and conditional independence assumptions. The second example illustrated how to evaluate the added value of a new risk factor using the NRI measure. Using the new risk classification provides greater precision in risk prediction, and in the example, the strategy using the new risk classification with treatment for selected individuals led to more favorable cost-effectiveness results.

Conclusions

These innovative methods for handling multiple diagnostic tests have improved the methodology within the field and should be adopted to provide more accurate estimates within cost-effectiveness analyses.

Highlights

Economic evaluations of multiple diagnostic tests often apply problematic simplistic approaches, such as ignoring conditional dependence between 2 tests or assuming a perfect final test in the diagnostic pathway. We provided guidance on how to apply improved methods for economic modeling. We introduced methods to model conditional dependence between 2 imperfect tests. We used an example to illustrate how assumptions about perfect diagnostic test accuracy and conditional independence between tests affect cost-effectiveness. Compared with the results of the area under the receiver-operating-characteristic curve, net reclassification improvement has distinct advantages in measuring the added value of a new risk factor for model-based economic evaluation. Economic evaluations that appropriately account for the complexities of diagnostic test pathways can help decision makers ensure efficient use of resources.

Prevalence of current and past COVID-19 in Ohio adults

PURPOSE

To estimate the prevalence of current and past COVID-19 in Ohio adults.

METHODS

We used stratified, probability-proportionate-to-size cluster sampling. During July 2020, we enrolled 727 randomly-sampled adult English- and Spanish-speaking participants through a household survey. Participants provided nasopharyngeal swabs and blood samples to detect current and past COVID-19. We used Bayesian latent class models with multilevel regression and poststratification to calculate the adjusted prevalence of current and past COVID-19. We accounted for the potential effects of non-ignorable non-response bias.

RESULTS

The estimated statewide prevalence of current COVID-19 was 0.9% (95% credible interval: 0.1-2.0%), corresponding to ∼85,000 prevalent infections (95% credible interval: 6,300-177,000) in Ohio adults during the study period. The estimated statewide prevalence of past COVID-19 was 1.3% (95% credible interval: 0.2-2.7%), corresponding to ∼118,000 Ohio adults (95% credible interval: 22,000-240,000). Estimates did not change meaningfully due to non-response bias.

CONCLUSIONS

Total COVID-19 cases in Ohio in July 2020 were approximately 3.5 times as high as diagnosed cases. The lack of broad COVID-19 screening in the United States early in the pandemic resulted in a paucity of population-representative prevalence data, limiting the ability to measure the effects of statewide control efforts.

Asymptotic analysis of reliability measures for an imperfect dichotomous test

To access the reliability of a new dichotomous test and to capture the random variability of its results in the absence of a gold standard, two measures, the inconsistent acceptance probability (IAP) and inconsistent rejection probability (IRP), were introduced in the literature. In this paper, we first analyze the limiting behavior of both measures as the number of test repetitions increases and derive the corresponding accuracy estimates and rates of convergence. To overcome possible limitations of IRP and IAP, we then introduce a one-parameter family of refined reliability measures, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Delta (k, s)$$\end{document}Δ(k,s). Such measures characterize the consistency of the results of a dichotomous test in the absence of a gold standard as the threshold for a positive aggregate test result varies. Similar to IRP and IAP, we also derive corresponding accuracy estimates and rates of convergence for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Delta (k, s)$$\end{document}Δ(k,s) as the number k of test repetitions increases.

Accuracy of hospital-based surveillance systems for surgical site infection after adult spine surgery: A Bayesian latent class analysis

BACKGROUND

Surgical site infections (SSIs) of the spine are morbid and costly complications. An accurate surveillance system is required to properly describe the disease burden and the impact of interventions that mitigate SSI risk. Unfortunately, uniform approaches to conducting SSI surveillance are lacking because of varying SSI case definitions, the lack of a perfect reference case definition and heterogeneous data sources.

AIM

We assessed the accuracy of 4 independent data sources that capture SSIs after spine surgery, with estimation of a measurement error-adjusted SSI incidence.

METHODS

A Bayesian latent class model assessed the sensitivity/specificity of each data source to identify SSI and to estimate a measurement-error adjusted incidence. The four data sources used were: the discharge abstract database (DAD), the National Surgical Quality Improvement Program (NSQIP) database, the Infection Prevention and Control Canada (IPAC) database, and the Spine Adverse Events Severity database.

FINDINGS

A total of 904 patients underwent spine surgery in 2017. The most sensitive data source was DAD (0.799, 95% CrI 0.597, 0.943), while the least sensitive was NSQIP (0.497, 95% CrI 0.308, 0.694). The most specific data source was IPAC (0.997, 95% CrI 0.993, 1.000) and the least specific was DAD (0.969, 95% CrI 0.956, 0.981). The measurement error-adjusted SSI incidence was 0.030 (95% CrI 0.019, 0.045). The crude incidence using the DAD over-estimated the incidence, and the 3 other data sources under-estimated it.

CONCLUSION

SSI surveillance in the spine surgery population is feasible using several data sources, provided that measurement error is considered.

Estimating seroprevalence of SARS-CoV-2 in Ohio: A Bayesian multilevel poststratification approach with multiple diagnostic tests

Globally, severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) has infected more than 59 million people and killed more than 1.39 million. Designing and monitoring interventions to slow and stop the spread of the virus require knowledge of how many people have been and are currently infected, where they live, and how they interact. The first step is an accurate assessment of the population prevalence of past infections. There are very few population-representative prevalence studies of SARS-CoV-2 infections, and only two states in the United States-Indiana and Connecticut-have reported probability-based sample surveys that characterize statewide prevalence of SARS-CoV-2. One of the difficulties is the fact that tests to detect and characterize SARS-CoV-2 coronavirus antibodies are new, are not well characterized, and generally function poorly. During July 2020, a survey representing all adults in the state of Ohio in the United States collected serum samples and information on protective behavior related to SARS-CoV-2 and coronavirus disease 2019 (COVID-19). Several features of the survey make it difficult to estimate past prevalence: 1) a low response rate; 2) a very low number of positive cases; and 3) the fact that multiple poor-quality serological tests were used to detect SARS-CoV-2 antibodies. We describe a Bayesian approach for analyzing the biomarker data that simultaneously addresses these challenges and characterizes the potential effect of selective response. The model does not require survey sample weights; accounts for multiple imperfect antibody test results; and characterizes uncertainty related to the sample survey and the multiple imperfect, potentially correlated tests.

Estimation of diagnostic test accuracy: A "Rule of Three" for data with repeated observations but without a gold standard

This article considers how to estimate the accuracy of a diagnostic test when there are repeated observations, but without the availability of a gold standard or reference test. We identify conditions under which the structure of the observed data is rich enough to provide sufficient degrees of freedom, such that a suitable latent class model can be fitted with identifiable accuracy parameters. We show that a Rule of Three applies, specifying that accuracy can be evaluated as long as there are at least three observations per individual with the given test. This rule also applies if the three observations arise from combinations of different test methods, or from a sequential design in which individuals are tested for a maximum number of times with the same test but stopping if a positive (or negative) result occurs. The rule pertains to tests having an arbitrary number of response categories. Accuracy is evaluated by parameters reflecting rates of misclassification among the response categories, and the model also provides estimates of the underlying distribution of the true disease state. These ideas are illustrated by data from two medical studies. Issues discussed include the advantages and disadvantages of analyzing the response variable as binary or multinomial, as well as the feasibility of testing goodness of fit when the model incorporates a large number of parameters. Comparisons are possible between models that do or do not assume equal accuracy rates for the observations, and between models where certain misclassification parameters are or are not assumed to be zero.

Evaluation using latent class models of the diagnostic performances of three ELISA tests commercialized for the serological diagnosis of Coxiella burnetii infection in domestic ruminants

ELISA methods are the diagnostic tools recommended for the serological diagnosis of Coxiella burnetii infection in ruminants but their respective diagnostic performances are difficult to assess because of the absence of a gold standard. This study focused on three commercial ELISA tests with the following objectives (1) assess their sensitivity and specificity in sheep, goats and cattle, (2) assess the between- and within-herd seroprevalence distribution in these species, accounting for diagnostic errors, and (3) estimate optimal sample sizes considering sensitivity and specificity at herd level. We comparatively tested 1413 cattle, 1474 goat and 1432 sheep serum samples collected in France. We analyzed the cross-classified test results with a hierarchical zero-inflated beta-binomial latent class model considering each herd as a population and conditional dependence as a fixed effect. Potential biases and coverage probabilities of the model were assessed by simulation. Conditional dependence for truly seropositive animals was high in all species for two of the three ELISA methods. Specificity estimates were high, ranging from 94.8% [92.1; 97.8] to 99.2% [98.5; 99.7], whereas sensitivity estimates were generally low, ranging from 39.3 [30.7; 47.0] to 90.5% [83.3; 93.8]. Between- and within-herd seroprevalence estimates varied greatly among geographic areas and herds. Overall, goats showed higher within-herd seroprevalence levels than sheep and cattle. The optimal sample size maximizing both herd sensitivity and herd specificity varied from 3 to at least 20 animals depending on the test and ruminant species. This study provides better interpretation of three widely used commercial ELISA tests and will make it possible to optimize their implementation in future studies. The methodology developed may likewise be applied to other human or animal diseases.

Comparative diagnostic accuracy studies with an imperfect reference standard - a comparison of correction methods

BACKGROUND

Staquet et al. and Brenner both developed correction methods to estimate the sensitivity and specificity of a binary-response index test when the reference standard is imperfect and its sensitivity and specificity are known. However, to our knowledge, no study has compared the statistical properties of these methods, despite their long application in diagnostic accuracy studies.

AIM

To compare the correction methods developed by Staquet et al. and Brenner.

METHODS

Simulations techniques were employed to compare the methods under assumptions that the new test and the reference standard are conditionally independent or dependent given the true disease status of an individual. Three clinical datasets were analysed to understand the impact of using each method to inform clinical decision-making.

RESULTS

Under the assumption of conditional independence, the Staquet et al. correction method outperforms the Brenner correction method irrespective of the prevalence of disease and whether the performance of the reference standard is better or worse than the index test. However, when the prevalence of the disease is high (> 0.9) or low (< 0.1), the Staquet et al. correction method can produce illogical results (i.e. results outside [0,1]). Under the assumption of conditional dependence; both methods failed to estimate the sensitivity and specificity of the index test especially when the covariance terms between the index test and the reference standard is not close to zero.

CONCLUSION

When the new test and the imperfect reference standard are conditionally independent, and the sensitivity and specificity of the imperfect reference standard are known, the Staquet et al. correction method outperforms the Brenner method. However, where the prevalence of the target condition is very high or low or the two tests are conditionally dependent, other statistical methods such as latent class approaches should be considered.

Effect of sample size and length of observation period on the reliability of apparent pig organ lesion prevalence

Comprehensive identification of on-farm animal-health issues still requires extensive efforts so that in practice such monitoring is applied only sparsely. An appealing approach to improve on-farm animal health and welfare monitoring is the application of organ lesion scoring data from the abattoir as such is instantly available for every commercial farm in Europe. Unfortunately, it is also well-known that organ lesion scoring is often unreliable because results are altered by several non-health-related factors, diluting the validity of lesion scoring prevalence as a proxy for on-farm animal health. However, it is theoretically possible to improve prevalence reliability a-posteriori by application of time-series smoothing. The aim of this paper was therefore to analyse whether it is practically possible to increase apparent prevalence estimation reliability retrospectively using a running average, and, if so, which window length and smallest sample size should be preferred in such an approach. Because no gold standard for direct evaluation of lesion reliability is available for field-data, apparent prevalence reliability had to be approximated using prevalence agreement over time. Results indicate that by raising the number of lesion scores per prevalence estimate, apparent prevalence agreement over time can in general be considerably increased. Based on findings presented, a reasonable threshold for prevalence estimation is given by at least n = 50 lesions per farm/abattoir/time-series segment. Results further suggest that it is necessary to consider differences in prevalence sample size for future monitoring purposes, because prevalences that are estimated on a continuum of different sample sizes put together in one evaluation may induce substantial error in prevalence estimates.

Estimation of the sensitivity and specificity of four serum ELISA and one fecal PCR for diagnosis of paratuberculosis in adult dairy cattle in New Zealand using Bayesian latent class analysis

In New Zealand, a new diagnostic approach for the control of paratuberculosis in mixed aged milking cows has been developed using a combination of ELISA and quantitative fecal PCR (f-qPCR). Our analysis was designed to evaluate performance of these individual tests in infected or infectious mixed aged cows across the prevalence of infection typically encountered on NZ dairy farms and calculate test accuracy when used as a screening test of serological ELISAs for four separate antigens read in parallel followed by a confirmatory quantitative f-qPCR test. Data from a cross-sectional study of 20 moderate prevalence herds was combined with existing data from 2 low and 20 high prevalence herds forming a dataset of 3845 paired serum and fecal samples. Incidence of clinical Johne's disease (JD) was used to classify herds into three prevalence categories. High (≥ 3% annual clinical JD for the last three years), moderate (<3 - 1%) and low (<1% incidence for at least the last five years). Positive tests were declared if> 50 ELISA units and f-qPCR at two cut-points (≥1 × 10⁴ genomes/mL or >1 × 10³ genomes/mL). Fixed Bayesian latent class models at both f-qPCR cut-points, accounted for conditional independence and paired conditional dependence. Mixed models at both f-qPCR cut-points, using a different mechanism to account for conditional dependencies between tests were also implemented. Models (24 in number) were constructed using OpenBUGS. The aim was to identify Mycobacterium avium subsp. paratuberculosis (MAP) infected cows that met at least one of two criteria: shedding sufficient MAP in feces to be detected by f-qPCR or mounting a detectable MAP antibody response. The best fit to the data was obtained by modelling pairwise dependencies between tests in a fixed model or by accounting for dependencies in a mixed model at a fecal cut-off of ≥1 × 10⁴ genomes/mL. Test performance differed with prevalence, but models were robust to prior assumptions. For the fixed model, at a prevalence of 0.29 (95 % probability interval (PI) = 0.25-0.33), as a screening plus confirmatory f-qPCR, post-test probability for disease in a positive animal was 0.84 (95 %PI = 0.80-0.88) and 0.16 (95 %PI = 0.15-0.18) for disease in a test negative animal. In low prevalence herds (0.01(95 %PI = 0.00-0.04)) the equivalent figures were 0.84 (95 %PI = 0.08-0.92) and 0.00 (95 %PI = 0.00-0.02). These results suggest this is a useful tool to control JD on dairy farms, particularly in herds with higher levels of infection, where the sampling and testing cost per animal is defrayed across more detected animals.

Comparison between thoracic ultrasonography and thoracic radiography for the detection of thoracic lesions in dairy calves using a two-stage Bayesian method

Infectious bronchopneumonia is a lower respiratory tract disease with major economic consequences in dairy calves. Thoracic radiography (TR) and thoracic ultrasonography (TUS) are two imaging diagnostic procedures available in bovine medicine for identifying thoracic lesions. However, no study has investigated whether one of these tests is superior to the other or if they provide comparable results for the detection of thoracic lesions in calves. The objective of this study was therefore to estimate and to compare the performances of TUS and TR for the detection of thoracic lesions in dairy calves. A prospective cross-sectional study was performed in a hospital setting. A total of 50 calves (≥7 days old; ≤100 kg; standing; pCO2 ≥ 53 mmHg; any reason of presentation) were enrolled. Every calf underwent TUS and TR. Only calves with thoracic lesions on TUS and/or TR were controlled by thoracic computed tomography (CT) (the gold standard). Calves without lesions were not controlled by CT. A two-stage Bayesian framework was used. The sensitivities (Se) and specificities (Sp) of both tests individually and used in series or parallel were estimated. The Se and Sp of TUS were 0.81 (95 % BCI (Bayesian Credible Interval): 0.65; 0.92) and 0.90 (95 % BCI: 0.81; 0.96), respectively. The Se and Sp of TR were 0.86 (95 % BCI: 0.62; 0.99) and 0.89 (95 % BCI: 0.67; 0.99), respectively. This study did not reveal any differences between both tests. Using TUS and TR in series was more specific than using both tests in parallel. The performances of TUS alone were not different from the performances of both tests in series or in parallel. In conclusion, TUS and TR were equivalent in detecting thoracic lesions in this study. Using TUS alone allowed an accurate detection of thoracic lesions in dairy calves. Further studies enrolling a larger sample (> 400 calves) and allowing adequate power to be achieved would be necessary to confirm these results.