1
|
Keddie SH, Baerenbold O, Keogh RH, Bradley J. Estimating sensitivity and specificity of diagnostic tests using latent class models that account for conditional dependence between tests: a simulation study. BMC Med Res Methodol 2023; 23:58. [PMID: 36894883 PMCID: PMC9999546 DOI: 10.1186/s12874-023-01873-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/30/2022] [Accepted: 02/20/2023] [Indexed: 03/11/2023] Open
Abstract
BACKGROUND Latent class models are increasingly used to estimate the sensitivity and specificity of diagnostic tests in the absence of a gold standard, and are commonly fitted using Bayesian methods. These models allow us to account for 'conditional dependence' between two or more diagnostic tests, meaning that the results from tests are correlated even after conditioning on the person's true disease status. The challenge is that it is not always clear to researchers whether conditional dependence exists between tests and whether it exists in all or just some latent classes. Despite the increasingly widespread use of latent class models to estimate diagnostic test accuracy, the impact of the conditional dependence structure chosen on the estimates of sensitivity and specificity remains poorly investigated. METHODS A simulation study and a reanalysis of a published case study are used to highlight the impact of the conditional dependence structure chosen on estimates of sensitivity and specificity. We describe and implement three latent class random-effect models with differing conditional dependence structures, as well as a conditional independence model and a model that assumes perfect test accuracy. We assess the bias and coverage of each model in estimating sensitivity and specificity across different data generating mechanisms. RESULTS The findings highlight that assuming conditional independence between tests within a latent class, where conditional dependence exists, results in biased estimates of sensitivity and specificity and poor coverage. The simulations also reiterate the substantial bias in estimates of sensitivity and specificity when incorrectly assuming a reference test is perfect. The motivating example of tests for Melioidosis highlights these biases in practice with important differences found in estimated test accuracy under different model choices. CONCLUSIONS We have illustrated that misspecification of the conditional dependence structure leads to biased estimates of sensitivity and specificity when there is a correlation between tests. Due to the minimal loss in precision seen by using a more general model, we recommend accounting for conditional dependence even if researchers are unsure of its presence or it is only expected at minimal levels.
Collapse
Affiliation(s)
- Suzanne H Keddie
- Department of Infectious Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, UK.
| | - Oliver Baerenbold
- Department of Infectious Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, UK
| | - Ruth H Keogh
- Department of Medical Statistics, London School of Hygiene and Tropical Medicine, London, UK
| | - John Bradley
- Department of Infectious Disease Epidemiology, London School of Hygiene and Tropical Medicine, London, UK
- MRC International Statistics and Epidemiology Group, London School of Hygiene and Tropical Medicine, London, UK
| |
Collapse
|
2
|
Goh GS, Lonner JH. Response to Letter to the Editor on "The Paradox of Patient-Reported Outcome Measures: Should We Prioritize "Feeling Better" or "Feeling Good" After Total Knee Arthroplasty?". J Arthroplasty 2022; 37:e10-e11. [PMID: 36162930 DOI: 10.1016/j.arth.2022.05.030] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Accepted: 05/13/2022] [Indexed: 02/02/2023] Open
Affiliation(s)
- Graham S Goh
- Rothman Orthopaedic Institute at Thomas Jefferson University, Philadelphia, Pennsylvania
| | - Jess H Lonner
- Rothman Orthopaedic Institute at Thomas Jefferson University, Philadelphia, Pennsylvania
| |
Collapse
|
3
|
Duncan DB, Mackett K, Ali MU, Yamamura D, Balion C. Performance of saliva compared with nasopharyngeal swab for diagnosis of COVID-19 by NAAT in cross-sectional studies: Systematic review and meta-analysis. Clin Biochem 2022; 117:84-93. [PMID: 35952732 PMCID: PMC9359767 DOI: 10.1016/j.clinbiochem.2022.08.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/22/2022] [Revised: 07/14/2022] [Accepted: 08/05/2022] [Indexed: 11/03/2022]
Abstract
Nucleic acid amplification testing (NAAT) is the preferred method to diagnose coronavirus disease 2019 (COVID-19). Saliva has been suggested as an alternative to nasopharyngeal swabs (NPS), but previous systematic reviews were limited by the number and types of studies available. The objective of this systematic review and meta-analysis was to assess the diagnostic performance of saliva compared with NPS for COVID-19. We searched Ovid MEDLINE, Embase, Cochrane, and Scopus databases up to 24 April 2021 for studies that directly compared paired NPS and saliva specimens taken at the time of diagnosis. Meta-analysis was performed using an exact binomial rendition of the bivariate mixed-effects regression model. Risk of bias was assessed using the QUADAS-2 tool. Of 2683 records, we included 23 studies with 25 cohorts, comprising 11,582 paired specimens. A wide variety of NAAT assays and collection methods were used. Meta-analysis gave a pooled sensitivity of 87 % (95 % CI = 83-90 %) and specificity of 99 % (95 % CI = 98-99 %). Subgroup analyses showed the highest sensitivity when the suspected individual is tested in an outpatient setting and is symptomatic. Our results support the use of saliva NAAT as an alternative to NPS NAAT for the diagnosis of COVID-19.
Collapse
Affiliation(s)
- Donald Brody Duncan
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario L8S 4K1, Canada; Microbiology Department, Hamilton Regional Laboratory Medicine Program, Hamilton Health Sciences, Hamilton, Ontario L8L 2X2, Canada
| | - Katharine Mackett
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario L8S 4K1, Canada
| | - Muhammad Usman Ali
- Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario L8S 4K1, Canada
| | - Deborah Yamamura
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario L8S 4K1, Canada; Microbiology Department, Hamilton Regional Laboratory Medicine Program, Hamilton Health Sciences, Hamilton, Ontario L8L 2X2, Canada; Division of Infectious Diseases, Department of Medicine, McMaster University, Hamilton, Ontario L8V 1C3, Canada
| | - Cynthia Balion
- Department of Pathology and Molecular Medicine, McMaster University, Hamilton, Ontario L8S 4K1, Canada; Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario L8S 4K1, Canada.
| |
Collapse
|
4
|
MacLean EL, Kohli M, Köppel L, Schiller I, Sharma SK, Pai M, Denkinger CM, Dendukuri N. Bayesian latent class analysis produced diagnostic accuracy estimates that were more interpretable than composite reference standards for extrapulmonary tuberculosis tests. Diagn Progn Res 2022; 6:11. [PMID: 35706064 PMCID: PMC9202094 DOI: 10.1186/s41512-022-00125-x] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/11/2021] [Accepted: 03/30/2022] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND Evaluating the accuracy of extrapulmonary tuberculosis (TB) tests is challenging due to lack of a gold standard. Latent class analysis (LCA), a statistical modeling approach, can adjust for reference tests' imperfect accuracies to produce less biased test accuracy estimates than those produced by commonly used methods like composite reference standards (CRSs). Our objective is to illustrate how Bayesian LCA can address the problem of an unavailable gold standard and demonstrate how it compares to using CRSs for extrapulmonary TB tests. METHODS We re-analyzed a dataset of presumptive extrapulmonary TB cases in New Delhi, India, for three forms of extrapulmonary TB. Results were available for culture, smear microscopy, Xpert MTB/RIF, and a non-microbiological test, cytopathology/histopathology, or adenosine deaminase (ADA). A diagram was used to define assumed relationships between observed tests and underlying latent variables in the Bayesian LCA with input from an inter-disciplinary team. We compared the results to estimates obtained from a sequence of CRSs defined by increasing numbers of positive reference tests necessary for positive disease status. RESULTS Data were available from 298, 388, and 230 individuals with presumptive TB lymphadenitis, meningitis, and pleuritis, respectively. Using Bayesian LCA, estimates were obtained for accuracy of all tests and for extrapulmonary TB prevalence. Xpert sensitivity neared that of culture for TB lymphadenitis and meningitis but was lower for TB pleuritis, and specificities of all microbiological tests approached 100%. Non-microbiological tests' sensitivities were high, but specificities were only moderate, preventing disease rule-in. CRSs' only provided estimates of Xpert and these varied widely per CRS definition. Accuracy of the CRSs also varied by definition, and no CRS was 100% accurate. CONCLUSION Unlike CRSs, Bayesian LCA takes into account known information about test performance resulting in accuracy estimates that are easier to interpret. LCA should receive greater consideration for evaluating extrapulmonary TB diagnostic tests.
Collapse
Affiliation(s)
- Emily L MacLean
- McGill International TB Centre, Research Institute of the McGill University Health Centre, Montréal, Canada
- Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montréal, Canada
| | | | - Lisa Köppel
- Division of Tropical Medicine, Center of Infectious Diseases, Heidelberg University, Heidelberg, Germany
| | - Ian Schiller
- Department of Medicine, McGill University Health Centre, Montréal, Canada
| | - Surendra K Sharma
- Department of Medicine, All India Institute of Medical Sciences, New Delhi, India
| | - Madhukar Pai
- McGill International TB Centre, Research Institute of the McGill University Health Centre, Montréal, Canada
- Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montréal, Canada
| | - Claudia M Denkinger
- McGill International TB Centre, Research Institute of the McGill University Health Centre, Montréal, Canada
- Division of Tropical Medicine, Center of Infectious Diseases, Heidelberg University, Heidelberg, Germany
| | - Nandini Dendukuri
- McGill International TB Centre, Research Institute of the McGill University Health Centre, Montréal, Canada.
- Department of Medicine, McGill University Health Centre, Montréal, Canada.
| |
Collapse
|
5
|
Drew C, Badio M, Dennis D, Hensley L, Higgs E, Sneller M, Fallah M, Reilly C. Simplifying the estimation of diagnostic testing accuracy over time for high specificity tests in the absence of a gold standard. Biometrics 2022. [PMID: 35531799 DOI: 10.1111/biom.13689] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Accepted: 04/29/2022] [Indexed: 11/29/2022]
Abstract
Many different methods for evaluating diagnostic test results in the absence of a gold standard have been proposed. In this paper, we discuss how one common method, a maximum likelihood estimate for a latent class model found via the Expectation-Maximization (EM) algorithm can be applied to longitudinal data where test sensitivity changes over time. We also propose two simplified and nonparametric methods which use data-based indicator variables for disease status and compare their accuracy to the maximum likelihood estimation (MLE) results. We find that with high specificity tests, the performance of simpler approximations may be just as high as the MLE.
Collapse
Affiliation(s)
- Clara Drew
- Division of Biostatistics, University of Minnesota, Minneapolis, Minnesota, USA
| | - Moses Badio
- Partnership for Research on Vaccines and Infectious Diseases in Liberia (PREVAIL), Monrovia, Liberia.,Department of Epidemiology and Biostatistics, University of California, San Francisco, California, USA
| | - Dehkontee Dennis
- Partnership for Research on Vaccines and Infectious Diseases in Liberia (PREVAIL), Monrovia, Liberia
| | - Lisa Hensley
- Partnership for Research on Vaccines and Infectious Diseases in Liberia (PREVAIL), Monrovia, Liberia.,Division of Clinical Research, National Institute of Allergy and Infectious Diseases, Bethesda, Maryland, USA
| | - Elizabeth Higgs
- Partnership for Research on Vaccines and Infectious Diseases in Liberia (PREVAIL), Monrovia, Liberia.,Division of Clinical Research, National Institute of Allergy and Infectious Diseases, Bethesda, Maryland, USA
| | - Michael Sneller
- Laboratory of Immunoregulation, National Institute of Allergy and Infectious Diseases, Bethesda, Maryland, USA
| | - Mosoka Fallah
- Partnership for Research on Vaccines and Infectious Diseases in Liberia (PREVAIL), Monrovia, Liberia.,National Public Health Institute of Liberia, Monrovia, Liberia
| | - Cavan Reilly
- Division of Biostatistics, University of Minnesota, Minneapolis, Minnesota, USA.,Partnership for Research on Vaccines and Infectious Diseases in Liberia (PREVAIL), Monrovia, Liberia
| |
Collapse
|
6
|
Sykes JE, Reagan KL, Nally JE, Galloway RL, Haake DA. Role of Diagnostics in Epidemiology, Management, Surveillance, and Control of Leptospirosis. Pathogens 2022; 11:395. [PMID: 35456070 PMCID: PMC9032781 DOI: 10.3390/pathogens11040395] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Revised: 03/22/2022] [Accepted: 03/22/2022] [Indexed: 12/12/2022] Open
Abstract
A One Health approach to the epidemiology, management, surveillance, and control of leptospirosis relies on accessible and accurate diagnostics that can be applied to humans and companion animals and livestock. Diagnosis should be multifaceted and take into account exposure risk, clinical presentation, and multiple direct and/or indirect diagnostic approaches. Methods of direct detection of Leptospira spp. include culture, histopathology and immunostaining of tissues or clinical specimens, and nucleic acid amplification tests (NAATs). Indirect serologic methods to detect leptospiral antibodies include the microscopic agglutination test (MAT), the enzyme-linked immunosorbent assay (ELISA), and lateral flow methods. Rapid diagnostics that can be applied at the point-of-care; NAAT and lateral flow serologic tests are essential for management of acute infection and control of outbreaks. Culture is essential to an understanding of regional knowledge of circulating strains, and we discuss recent improvements in methods for cultivation, genomic sequencing, and serotyping. We review the limitations of NAATs, MAT, and other diagnostic approaches in the context of our expanding understanding of the diversity of pathogenic Leptospira spp. Novel approaches are needed, such as loop mediated isothermal amplification (LAMP) and clustered regularly interspaced short palindromic repeats (CRISPR)-based approaches to leptospiral nucleic acid detection.
Collapse
|
7
|
Schofield MR, Maze MJ, Crump JA, Rubach MP, Galloway RL, Sharples KJ. Rejoinder to "On the robustness of latent class models for diagnostic testing with no gold standard". Stat Med 2021; 40:4770-4771. [PMID: 34515367 DOI: 10.1002/sim.9157] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/26/2021] [Accepted: 07/26/2021] [Indexed: 11/10/2022]
Affiliation(s)
- Matthew R Schofield
- Department of Mathematics and Statistics, University of Otago, Dunedin, New Zealand
| | - Michael J Maze
- Centre for International Health, University of Otago, Dunedin, New Zealand.,Department of Medicine, University of Otago, Christchurch, Christchurch, New Zealand
| | - John A Crump
- Department of Preventive and Social Medicine, University of Otago, Dunedin, New Zealand
| | - Matthew P Rubach
- Division of Infectious Diseases, Duke University, Durham, North Carolina, USA
| | - Renee L Galloway
- Bacterial Special Pathogens Branch, US Center for Disease Control and Prevention, Atlanta, Georgia, USA
| | - Katrina J Sharples
- Department of Mathematics and Statistics, University of Otago, Dunedin, New Zealand
| |
Collapse
|
8
|
Dendukuri N. Commentary on "On the robustness of latent class models for diagnostic testing with no gold-standard" by Schofield et al. Stat Med 2021; 40:4766-4769. [PMID: 34515365 DOI: 10.1002/sim.9086] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2021] [Accepted: 05/21/2021] [Indexed: 11/10/2022]
Affiliation(s)
- Nandini Dendukuri
- Centre for Outcomes Research, McGill University Health Centre- Research Institute, Montreal, Quebec, Canada
| |
Collapse
|