1
|
Macdonald T, Dinnes J, Maniatopoulos G, Taylor-Phillips S, Shinkins B, Hogg J, Dunbar JK, Solebo AL, Sutton H, Attwood J, Pogose M, Given-Wilson R, Greaves F, Macrae C, Pearson R, Bamford D, Tufail A, Liu X, Denniston AK. Target Product Profile for a Machine Learning-Automated Retinal Imaging Analysis Software for Use in English Diabetic Eye Screening: Protocol for a Mixed Methods Study. JMIR Res Protoc 2024; 13:e50568. [PMID: 38536234 PMCID: PMC11007610 DOI: 10.2196/50568] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2023] [Revised: 02/02/2024] [Accepted: 02/13/2024] [Indexed: 04/13/2024] Open
Abstract
BACKGROUND Diabetic eye screening (DES) represents a significant opportunity for the application of machine learning (ML) technologies, which may improve clinical and service outcomes. However, successful integration of ML into DES requires careful product development, evaluation, and implementation. Target product profiles (TPPs) summarize the requirements necessary for successful implementation so these can guide product development and evaluation. OBJECTIVE This study aims to produce a TPP for an ML-automated retinal imaging analysis software (ML-ARIAS) system for use in DES in England. METHODS This work will consist of 3 phases. Phase 1 will establish the characteristics to be addressed in the TPP. A list of candidate characteristics will be generated from the following sources: an overview of systematic reviews of diagnostic test TPPs; a systematic review of digital health TPPs; and the National Institute for Health and Care Excellence's Evidence Standards Framework for Digital Health Technologies. The list of characteristics will be refined and validated by a study advisory group (SAG) made up of representatives from key stakeholders in DES. This includes people with diabetes; health care professionals; health care managers and leaders; and regulators and policy makers. In phase 2, specifications for these characteristics will be drafted following a series of semistructured interviews with participants from these stakeholder groups. Data collected from these interviews will be analyzed using the shortlist of characteristics as a framework, after which specifications will be drafted to create a draft TPP. Following approval by the SAG, in phase 3, the draft will enter an internet-based Delphi consensus study with participants sought from the groups previously identified, as well as ML-ARIAS developers, to ensure feasibility. Participants will be invited to score characteristic and specification pairs on a scale from "definitely exclude" to "definitely include," and suggest edits. The document will be iterated between rounds based on participants' feedback. Feedback on the draft document will be sought from a group of ML-ARIAS developers before its final contents are agreed upon in an in-person consensus meeting. At this meeting, representatives from the stakeholder groups previously identified (minus ML-ARIAS developers, to avoid bias) will be presented with the Delphi results and feedback of the user group and asked to agree on the final contents by vote. RESULTS Phase 1 was completed in November 2023. Phase 2 is underway and expected to finish in March 2024. Phase 3 is expected to be complete in July 2024. CONCLUSIONS The multistakeholder development of a TPP for an ML-ARIAS for use in DES in England will help developers produce tools that serve the needs of patients, health care providers, and their staff. The TPP development process will also provide methods and a template to produce similar documents in other disease areas. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID) DERR1-10.2196/50568.
Collapse
Affiliation(s)
- Trystan Macdonald
- Ophthalmology Department, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham National Health Service Foundation Trust, Birmingham, United Kingdom
- Academic Unit of Ophthalmology, Institute of Inflammation and Aging, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, Birmingham, United Kingdom
| | - Jacqueline Dinnes
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, Birmingham, United Kingdom
| | | | | | - Bethany Shinkins
- Warwick Medical School, University of Warwick, Coventry, United Kingdom
| | - Jeffry Hogg
- Population Health Sciences Institute, Faculty of Medical Sciences, The University of Newcastle upon Tyne, Newcastle, United Kingdom
| | | | - Ameenat Lola Solebo
- Population Policy and Practice, University College London Great Ormond Street Institute of Child Health, London, United Kingdom
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
| | | | - John Attwood
- Alder Hey Children's Hospital, Alder Hey Children's Hospital NHS Foundation Trust, Liverpool, United Kingdom
| | | | - Rosalind Given-Wilson
- St. George's University Hospitals National Health Service Foundation Trust, London, United Kingdom
| | - Felix Greaves
- National Institute for Health and Care Excellence, London, United Kingdom
- Faculty of Medicine, School of Public Health, Imperial College London, London, United Kingdom
| | - Carl Macrae
- Nottingham University Business School, University of Nottingham, Nottingham, United Kingdom
| | - Russell Pearson
- Medicines and Healthcare Products Regulatory Agency, London, United Kingdom
| | | | - Adnan Tufail
- Moorfields Eye Hospital NHS Foundation Trust, London, United Kingdom
- Institute of Ophthalmology, University College London, London, United Kingdom
| | - Xiaoxuan Liu
- Ophthalmology Department, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham National Health Service Foundation Trust, Birmingham, United Kingdom
- Academic Unit of Ophthalmology, Institute of Inflammation and Aging, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, Birmingham, United Kingdom
| | - Alastair K Denniston
- Ophthalmology Department, Queen Elizabeth Hospital Birmingham, University Hospitals Birmingham National Health Service Foundation Trust, Birmingham, United Kingdom
- Academic Unit of Ophthalmology, Institute of Inflammation and Aging, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom
- National Institute for Health and Care Research Birmingham Biomedical Research Centre, Birmingham, United Kingdom
- Centre for Regulatory Science and Innovation, Birmingham Health Partners, Birmingham, United Kingdom
- National Institute for Health and Care Research Biomedical Research Centre at Moorfields and University College London Institute of Ophthalmology, London, United Kingdom
| |
Collapse
|
2
|
Bigio J, MacLean ELH, Das R, Sulis G, Kohli M, Berhane S, Dinnes J, Deeks JJ, Brümmer LE, Denkinger CM, Pai M. Accuracy of package inserts of SARS-CoV-2 rapid antigen tests: a secondary analysis of manufacturer versus systematic review data. Lancet Microbe 2023; 4:e875-e882. [PMID: 37844595 DOI: 10.1016/s2666-5247(23)00222-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 10/14/2022] [Revised: 05/05/2023] [Accepted: 07/17/2023] [Indexed: 10/18/2023]
Abstract
BACKGROUND Rapid antigen tests (RATs) were crucial during the COVID-19 pandemic. Information provided by the test manufacturer in product package inserts, also known as instructions for use (IFUs), is often the only data available to clinicians, public health professionals, and individuals on the diagnostic accuracy of these tests. We aimed to assess whether manufacturer IFU accuracy data aligned with evidence from independent research. METHODS We searched company websites for package inserts for RATs that were included in the July 2022 update of the Cochrane meta-analysis of SARS-CoV-2 RATs, which served as a benchmark for research evidence. We fitted bivariate hierarchical models to obtain absolute differences in sensitivity and specificity between IFU and Cochrane Review estimates for each test, as well as overall combined differences. FINDINGS We found 22 (100%) of 22 IFUs of the RATs included in the Cochrane Review. IFUs for 12 (55%) of 22 RATs reported statistically significantly higher sensitivity estimates than the Cochrane Review, and none reported lower estimates. The mean difference between IFU and Cochrane Review sensitivity estimates across tests was 12·0% (95% CI 7·5-16·6). IFUs in three (14%) of 22 diagnostic tests had significantly higher specificity estimates than the Cochrane Review and two (9%) of 22 had lower estimates. The mean difference between IFU and Cochrane Review specificity estimates across tests was 0·3% (95% CI 0·1-0·5). If 100 people with SARS-CoV-2 infection were tested with each of the tests in this study, on average 12 fewer people would be correctly diagnosed than is suggested by the package inserts. INTERPRETATION Health professionals and the public should be aware that package inserts for SARS-CoV-2 RATs might provide an overly optimistic picture of the sensitivity of a test. Regulatory bodies should strengthen their requirements for the reporting of diagnostic accuracy data in package inserts and policy makers should demand independent validation data for decision making. FUNDING None.
Collapse
Affiliation(s)
- Jacob Bigio
- Research Institute of the McGill University Health Centre, Montreal General Hospital, Montreal, QC, Canada; McGill International TB Centre, Montreal, QC, Canada.
| | - Emily L-H MacLean
- Central Clinical School, Faculty of Medicine and Health, The University of Sydney, Sydney, NSW, Australia
| | - Rishav Das
- Research Institute of the McGill University Health Centre, Montreal General Hospital, Montreal, QC, Canada; McGill International TB Centre, Montreal, QC, Canada
| | - Giorgia Sulis
- School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, ON, Canada; Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada
| | | | - Sarah Berhane
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Lukas E Brümmer
- Division of Infectious Diseases and Tropical Medicine, Center of Infectious Diseases, Heidelberg University Hospital, Germany
| | - Claudia M Denkinger
- Division of Infectious Diseases and Tropical Medicine, Center of Infectious Diseases, Heidelberg University Hospital, Germany; German Centre for Infection Research, Partner Site Heidelberg University Hospital, Heidelberg, Germany
| | - Madhukar Pai
- McGill International TB Centre, Montreal, QC, Canada; Department of Epidemiology, Biostatistics and Occupational Health, School of Population and Global Health, Faculty of Medicine and Health Sciences, McGill University, Montreal, QC, Canada
| |
Collapse
|
3
|
Matin RN, Dinnes J. Diagnosis of suspicious pigmented lesions in specialist settings with artificial intelligence. Lancet Digit Health 2023; 5:e639-e640. [PMID: 37775185 DOI: 10.1016/s2589-7500(23)00180-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/29/2023] [Accepted: 09/11/2023] [Indexed: 10/01/2023]
Affiliation(s)
- Rubeta N Matin
- Department of Dermatology, Oxford University Hospitals NHS Foundation Trust, Oxford OX3 7LE, UK.
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| |
Collapse
|
4
|
Kelly L, Coote L, Dinnes J, Fleming C, Holmes H, Matin RN. Key issues when considering adopting a skin cancer diagnostic tool which uses artificial intelligence. Br J Dermatol 2023:7115368. [PMID: 37041689 DOI: 10.1093/bjd/ljad080] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/20/2022] [Revised: 03/14/2023] [Accepted: 03/15/2023] [Indexed: 04/13/2023]
Affiliation(s)
- Laura Kelly
- York Health Economics Consortium, Enterprise House, Innovation Way, University of York, Heslington, York, UK
| | - Laura Coote
- York Health Economics Consortium, Enterprise House, Innovation Way, University of York, Heslington, York, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | | | - Hayden Holmes
- York Health Economics Consortium, Enterprise House, Innovation Way, University of York, Heslington, York, UK
| | - Rubeta N Matin
- Oxford University Hospitals NHS Foundation Trust, Churchill Hospital, Old Road, Headington, Oxford, UK
| |
Collapse
|
5
|
Veroniki AA, Tricco AC, Watt J, Tsokani S, Khan PA, Soobiah C, Negm A, Doherty-Kirby A, Taylor P, Lunny C, McGowan J, Little J, Mallon P, Moher D, Wong S, Dinnes J, Takwoingi Y, Saxinger L, Chan A, Isaranuwatchai W, Lander B, Meyers A, Poliquin G, Straus SE. Rapid antigen-based and rapid molecular tests for the detection of SARS-CoV-2: a rapid review with network meta-analysis of diagnostic test accuracy studies. BMC Med 2023; 21:110. [PMID: 36978074 PMCID: PMC10049780 DOI: 10.1186/s12916-023-02810-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Accepted: 02/27/2023] [Indexed: 03/30/2023] Open
Abstract
BACKGROUND The global spread of COVID-19 created an explosion in rapid tests with results in < 1 hour, but their relative performance characteristics are not fully understood yet. Our aim was to determine the most sensitive and specific rapid test for the diagnosis of SARS-CoV-2. METHODS Design: Rapid review and diagnostic test accuracy network meta-analysis (DTA-NMA). ELIGIBILITY CRITERIA Randomized controlled trials (RCTs) and observational studies assessing rapid antigen and/or rapid molecular test(s) to detect SARS-CoV-2 in participants of any age, suspected or not with SARS-CoV-2 infection. INFORMATION SOURCES Embase, MEDLINE, and Cochrane Central Register of Controlled Trials, up to September 12, 2021. OUTCOME MEASURES Sensitivity and specificity of rapid antigen and molecular tests suitable for detecting SARS-CoV-2. Data extraction and risk of bias assessment: Screening of literature search results was conducted by one reviewer; data abstraction was completed by one reviewer and independently verified by a second reviewer. Risk of bias was not assessed in the included studies. DATA SYNTHESIS Random-effects meta-analysis and DTA-NMA. RESULTS We included 93 studies (reported in 88 articles) relating to 36 rapid antigen tests in 104,961 participants and 23 rapid molecular tests in 10,449 participants. Overall, rapid antigen tests had a sensitivity of 0.75 (95% confidence interval 0.70-0.79) and specificity of 0.99 (0.98-0.99). Rapid antigen test sensitivity was higher when nasal or combined samples (e.g., combinations of nose, throat, mouth, or saliva samples) were used, but lower when nasopharyngeal samples were used, and in those classified as asymptomatic at the time of testing. Rapid molecular tests may result in fewer false negatives than rapid antigen tests (sensitivity: 0.93, 0.88-0.96; specificity: 0.98, 0.97-0.99). The tests with the highest sensitivity and specificity estimates were the Xpert Xpress rapid molecular test by Cepheid (sensitivity: 0.99, 0.83-1.00; specificity: 0.97, 0.69-1.00) among the 23 commercial rapid molecular tests and the COVID-VIRO test by AAZ-LMB (sensitivity: 0.93, 0.48-0.99; specificity: 0.98, 0.44-1.00) among the 36 rapid antigen tests we examined. CONCLUSIONS Rapid molecular tests were associated with both high sensitivity and specificity, while rapid antigen tests were mainly associated with high specificity, according to the minimum performance requirements by WHO and Health Canada. Our rapid review was limited to English, peer-reviewed published results of commercial tests, and study risk of bias was not assessed. A full systematic review is required. REVIEW REGISTRATION PROSPERO CRD42021289712.
Collapse
Affiliation(s)
- Areti Angeliki Veroniki
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada.
- Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada.
| | - Andrea C Tricco
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
- Epidemiology Division & Institute of Health Policy, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada
- Queen's Collaboration for Health Care Quality: A JBI Centre of Excellence, Kingston, Canada
| | - Jennifer Watt
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
| | - Sofia Tsokani
- School of Education, University of Ioannina, Ioannina, Greece
| | - Paul A Khan
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
| | - Charlene Soobiah
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
- Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada
| | - Ahmed Negm
- University of Alberta, Edmonton, AB, Canada
| | - Amanda Doherty-Kirby
- Patient Partner, Strategy for Patient Oriented-Research Evidence Alliance (SPOR EA), Toronto, Canada
| | - Paul Taylor
- Patient Partner, Strategy for Patient Oriented-Research Evidence Alliance (SPOR EA), Toronto, Canada
| | - Carole Lunny
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
| | - Jessie McGowan
- University of Ottawa/Université d'Ottawa, Ottawa, ON, Canada
| | - Julian Little
- University of Ottawa/Université d'Ottawa, Ottawa, ON, Canada
| | | | - David Moher
- Ottawa Hospital Research Institute/Institut de Recherche de L'Hôpital d'Ottawa, Ottawa, ON, Canada
| | - Sabrina Wong
- University of British Columbia, Vancouver, BC, Canada
| | | | | | | | | | | | - Bryn Lander
- Health Canada (Ottawa)/Santé Canada (Ottawa), Ottawa, ON, Canada
| | - Adrienne Meyers
- Public Health Agency of Canada/Agence de La Santé Publique du Canada, Ottawa, ON, Canada
| | - Guillaume Poliquin
- Public Health Agency of Canada/Agence de La Santé Publique du Canada, Ottawa, ON, Canada
| | - Sharon E Straus
- Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael's Hospital, Unity Health Toronto, 209 Victoria Street, East Building, Toronto, ON, M5B 1T8, Canada
- Institute for Health Policy, Management, and Evaluation, University of Toronto, Toronto, ON, Canada
- Department of Geriatric Medicine, University of Toronto, Toronto, ON, Canada
| |
Collapse
|
6
|
Stegeman I, Ochodo EA, Guleid F, Holtman GA, Yang B, Davenport C, Deeks JJ, Dinnes J, Dittrich S, Emperador D, Hoo L, Spijker R, Takwoingi Y, Van den Bruel A, Wang J, Langendam M, Verbakel JY, Leeflang MMG. Routine laboratory testing to determine if a patient has COVID-19. Emergencias 2022; 34:465-467. [PMID: 36625697] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [MESH Headings] [Grants] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Affiliation(s)
- Inge Stegeman
- Department of Otorhinolaryngology and Head and Neck Surgery, University Medical Center Utrecht, Utrecht, Países Bajos. Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Países Bajos. Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, Países Bajos
| | - Eleanor A Ochodo
- Centre for Evidence-based Health Care, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, África del Sud. Centre for Global Health Research, Kenya Medical Research Institute, Kisumu, Kenia
| | - Fatuma Guleid
- KEMRI-Wellcome Trust Research Programme, Nairobi, Kenia
| | - Gea A Holtman
- Department of General Practice, University of Groningen, University Medical Centre Groningen, Groningen, Países Bajos
| | - Bada Yang
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Países Bajos
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, RU. NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, RU
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, RU. NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, RU
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, RU. NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, RU
| | | | | | - Lotty Hoo
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Países Bajos
| | - René Spijker
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Países Bajos. Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Países Bajos
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, RU. NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, RU
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Bélgica
| | - Junfeng Wang
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, Países Bajos
| | - Miranda Langendam
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Países Bajos
| | - Jan Y Verbakel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Bélgica
| | - Mariska M G Leeflang
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Países Bajos
| |
Collapse
|
7
|
Fox T, Geppert J, Dinnes J, Scandrett K, Bigio J, Sulis G, Hettiarachchi D, Mathangasinghe Y, Weeratunga P, Wickramasinghe D, Bergman H, Buckley BS, Probyn K, Sguassero Y, Davenport C, Cunningham J, Dittrich S, Emperador D, Hooft L, Leeflang MM, McInnes MD, Spijker R, Struyf T, Van den Bruel A, Verbakel JY, Takwoingi Y, Taylor-Phillips S, Deeks JJ. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2022; 11:CD013652. [PMID: 36394900 PMCID: PMC9671206 DOI: 10.1002/14651858.cd013652.pub2] [Citation(s) in RCA: 19] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
BACKGROUND The diagnostic challenges associated with the COVID-19 pandemic resulted in rapid development of diagnostic test methods for detecting SARS-CoV-2 infection. Serology tests to detect the presence of antibodies to SARS-CoV-2 enable detection of past infection and may detect cases of SARS-CoV-2 infection that were missed by earlier diagnostic tests. Understanding the diagnostic accuracy of serology tests for SARS-CoV-2 infection may enable development of effective diagnostic and management pathways, inform public health management decisions and understanding of SARS-CoV-2 epidemiology. OBJECTIVES To assess the accuracy of antibody tests, firstly, to determine if a person presenting in the community, or in primary or secondary care has current SARS-CoV-2 infection according to time after onset of infection and, secondly, to determine if a person has previously been infected with SARS-CoV-2. Sources of heterogeneity investigated included: timing of test, test method, SARS-CoV-2 antigen used, test brand, and reference standard for non-SARS-CoV-2 cases. SEARCH METHODS The COVID-19 Open Access Project living evidence database from the University of Bern (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) was searched on 30 September 2020. We included additional publications from the Evidence for Policy and Practice Information and Co-ordinating Centre (EPPI-Centre) 'COVID-19: Living map of the evidence' and the Norwegian Institute of Public Health 'NIPH systematic and living map on COVID-19 evidence'. We did not apply language restrictions. SELECTION CRITERIA We included test accuracy studies of any design that evaluated commercially produced serology tests, targeting IgG, IgM, IgA alone, or in combination. Studies must have provided data for sensitivity, that could be allocated to a predefined time period after onset of symptoms, or after a positive RT-PCR test. Small studies with fewer than 25 SARS-CoV-2 infection cases were excluded. We included any reference standard to define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction tests (RT-PCR), clinical diagnostic criteria, and pre-pandemic samples). DATA COLLECTION AND ANALYSIS We use standard screening procedures with three reviewers. Quality assessment (using the QUADAS-2 tool) and numeric study results were extracted independently by two people. Other study characteristics were extracted by one reviewer and checked by a second. We present sensitivity and specificity with 95% confidence intervals (CIs) for each test and, for meta-analysis, we fitted univariate random-effects logistic regression models for sensitivity by eligible time period and for specificity by reference standard group. Heterogeneity was investigated by including indicator variables in the random-effects logistic regression models. We tabulated results by test manufacturer and summarised results for tests that were evaluated in 200 or more samples and that met a modification of UK Medicines and Healthcare products Regulatory Agency (MHRA) target performance criteria. MAIN RESULTS We included 178 separate studies (described in 177 study reports, with 45 as pre-prints) providing 527 test evaluations. The studies included 64,688 samples including 25,724 from people with confirmed SARS-CoV-2; most compared the accuracy of two or more assays (102/178, 57%). Participants with confirmed SARS-CoV-2 infection were most commonly hospital inpatients (78/178, 44%), and pre-pandemic samples were used by 45% (81/178) to estimate specificity. Over two-thirds of studies recruited participants based on known SARS-CoV-2 infection status (123/178, 69%). All studies were conducted prior to the introduction of SARS-CoV-2 vaccines and present data for naturally acquired antibody responses. Seventy-nine percent (141/178) of studies reported sensitivity by week after symptom onset and 66% (117/178) for convalescent phase infection. Studies evaluated enzyme-linked immunosorbent assays (ELISA) (165/527; 31%), chemiluminescent assays (CLIA) (167/527; 32%) or lateral flow assays (LFA) (188/527; 36%). Risk of bias was high because of participant selection (172, 97%); application and interpretation of the index test (35, 20%); weaknesses in the reference standard (38, 21%); and issues related to participant flow and timing (148, 82%). We judged that there were high concerns about the applicability of the evidence related to participants in 170 (96%) studies, and about the applicability of the reference standard in 162 (91%) studies. Average sensitivities for current SARS-CoV-2 infection increased by week after onset for all target antibodies. Average sensitivity for the combination of either IgG or IgM was 41.1% in week one (95% CI 38.1 to 44.2; 103 evaluations; 3881 samples, 1593 cases), 74.9% in week two (95% CI 72.4 to 77.3; 96 evaluations, 3948 samples, 2904 cases) and 88.0% by week three after onset of symptoms (95% CI 86.3 to 89.5; 103 evaluations, 2929 samples, 2571 cases). Average sensitivity during the convalescent phase of infection (up to a maximum of 100 days since onset of symptoms, where reported) was 89.8% for IgG (95% CI 88.5 to 90.9; 253 evaluations, 16,846 samples, 14,183 cases), 92.9% for IgG or IgM combined (95% CI 91.0 to 94.4; 108 evaluations, 3571 samples, 3206 cases) and 94.3% for total antibodies (95% CI 92.8 to 95.5; 58 evaluations, 7063 samples, 6652 cases). Average sensitivities for IgM alone followed a similar pattern but were of a lower test accuracy in every time slot. Average specificities were consistently high and precise, particularly for pre-pandemic samples which provide the least biased estimates of specificity (ranging from 98.6% for IgM to 99.8% for total antibodies). Subgroup analyses suggested small differences in sensitivity and specificity by test technology however heterogeneity in study results, timing of sample collection, and smaller sample numbers in some groups made comparisons difficult. For IgG, CLIAs were the most sensitive (convalescent-phase infection) and specific (pre-pandemic samples) compared to both ELISAs and LFAs (P < 0.001 for differences across test methods). The antigen(s) used (whether from the Spike-protein or nucleocapsid) appeared to have some effect on average sensitivity in the first weeks after onset but there was no clear evidence of an effect during convalescent-phase infection. Investigations of test performance by brand showed considerable variation in sensitivity between tests, and in results between studies evaluating the same test. For tests that were evaluated in 200 or more samples, the lower bound of the 95% CI for sensitivity was 90% or more for only a small number of tests (IgG, n = 5; IgG or IgM, n = 1; total antibodies, n = 4). More test brands met the MHRA minimum criteria for specificity of 98% or above (IgG, n = 16; IgG or IgM, n = 5; total antibodies, n = 7). Seven assays met the specified criteria for both sensitivity and specificity. In a low-prevalence (2%) setting, where antibody testing is used to diagnose COVID-19 in people with symptoms but who have had a negative PCR test, we would anticipate that 1 (1 to 2) case would be missed and 8 (5 to 15) would be falsely positive in 1000 people undergoing IgG or IgM testing in week three after onset of SARS-CoV-2 infection. In a seroprevalence survey, where prevalence of prior infection is 50%, we would anticipate that 51 (46 to 58) cases would be missed and 6 (5 to 7) would be falsely positive in 1000 people having IgG tests during the convalescent phase (21 to 100 days post-symptom onset or post-positive PCR) of SARS-CoV-2 infection. AUTHORS' CONCLUSIONS Some antibody tests could be a useful diagnostic tool for those in whom molecular- or antigen-based tests have failed to detect the SARS-CoV-2 virus, including in those with ongoing symptoms of acute infection (from week three onwards) or those presenting with post-acute sequelae of COVID-19. However, antibody tests have an increasing likelihood of detecting an immune response to infection as time since onset of infection progresses and have demonstrated adequate performance for detection of prior infection for sero-epidemiological purposes. The applicability of results for detection of vaccination-induced antibodies is uncertain.
Collapse
Affiliation(s)
- Tilly Fox
- Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool, UK
| | - Julia Geppert
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Katie Scandrett
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Jacob Bigio
- Research Institute of the McGill University Health Centre, Montreal, Canada
- McGill International TB Centre, Montreal, Canada
| | - Giorgia Sulis
- Department of Epidemiology, Biostatistics and Occupational Health, McGill University, Montreal, Canada
| | - Dineshani Hettiarachchi
- Department of Anatomy Genetics and Biomedical Informatics, Faculty of Medicine, University of Colombo, Colombo, Sri Lanka
| | - Yasith Mathangasinghe
- Department of Anatomy Genetics and Biomedical Informatics, Faculty of Medicine, University of Colombo, Colombo, Sri Lanka
- Australian Regenerative Medicine Institute, Monash University, Clayton, Australia
| | - Praveen Weeratunga
- Department of Clinical Medicine, Faculty of Medicine, University of Colombo, Colombo, Sri Lanka
| | | | | | - Brian S Buckley
- Cochrane Response, Cochrane, London, UK
- Department of Surgery, University of the Philippines, Manila, Philippines
| | | | | | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jane Cunningham
- Global Malaria Programme, World Health Organization, Geneva, Switzerland
| | | | | | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht , Netherlands
| | - Mariska Mg Leeflang
- Epidemiology and Data Science, Amsterdam UMC location University of Amsterdam, Amsterdam, Netherlands
- Amsterdam Public Health, Amsterdam, Netherlands
| | | | - René Spijker
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Thomas Struyf
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Jan Y Verbakel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Sian Taylor-Phillips
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| |
Collapse
|
8
|
Mallett S, Dinnes J, Takwoingi Y, de Ruffano LF. TOMAS-R: A template to identify and plan analysis for clinically important variation and multiplicity in diagnostic test accuracy systematic reviews. Diagn Progn Res 2022; 6:18. [PMID: 36131330 PMCID: PMC9494799 DOI: 10.1186/s41512-022-00131-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/17/2022] [Accepted: 06/21/2022] [Indexed: 11/30/2022] Open
Abstract
The Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy (DTA) provides guidance on important aspects of conducting a test accuracy systematic review. In this paper we present TOMAS-R (Template of Multiplicity and Analysis in Systematic Reviews), a structured template to use in conjunction with current Cochrane DTA guidance, to help identify complexities in the review question and to assist planning of data extraction and analysis when clinically important variation and multiplicity is present. Examples of clinically important variation and multiplicity could include differences in participants, index tests and test methods, target conditions and reference standards used to define them, study design and methodological quality. Our TOMAS-R template goes beyond the broad topic headings in current guidance that are sources of potential variation and multiplicity, by providing prompts for common sources of heterogeneity encountered from our experience of authoring over 100 reviews. We provide examples from two reviews to assist users. The TOMAS-R template adds value by supplementing available guidance for DTA reviews by providing a tool to facilitate discussions between methodologists, clinicians, statisticians and patient/public team members to identify the full breadth of review question complexities early in the process. The use of a structured set of prompting questions at the important stage of writing the protocol ensures clinical relevance as a main focus of the review, while allowing identification of key clinical components for data extraction and later analysis thereby facilitating a more efficient review process.
Collapse
Affiliation(s)
- Sue Mallett
- grid.83440.3b0000000121901201UCL Centre for Medical Imaging, University College London, London, W1W 7TY UK
| | - Jacqueline Dinnes
- grid.6572.60000 0004 1936 7486Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT UK
- grid.412563.70000 0004 0376 6589NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Yemisi Takwoingi
- grid.6572.60000 0004 1936 7486Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT UK
- grid.412563.70000 0004 0376 6589NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Lavinia Ferrante de Ruffano
- grid.6572.60000 0004 1936 7486Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, B15 2TT UK
- grid.5685.e0000 0004 1936 9668York Health Economics Consortium, Enterprise House, Innovation Way, University of York, York, YO10 5NQ UK
| |
Collapse
|
9
|
Dinnes J, Davenport C. COVID-19 rapid antigen testing strategies must be evaluated in intended use settings. Lancet Reg Health West Pac 2022; 25:100542. [PMID: 35845813 PMCID: PMC9278341 DOI: 10.1016/j.lanwpc.2022.100542] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Affiliation(s)
- Jacqueline Dinnes
- Corresponding author at: Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, B15 2TT, UK.
| | | |
Collapse
|
10
|
Dinnes J, Sharma P, Berhane S, van Wyk SS, Nyaaba N, Domen J, Taylor M, Cunningham J, Davenport C, Dittrich S, Emperador D, Hooft L, Leeflang MM, McInnes MD, Spijker R, Verbakel JY, Takwoingi Y, Taylor-Phillips S, Van den Bruel A, Deeks JJ. Rapid, point-of-care antigen tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev 2022; 7:CD013705. [PMID: 35866452 PMCID: PMC9305720 DOI: 10.1002/14651858.cd013705.pub3] [Citation(s) in RCA: 55] [Impact Index Per Article: 27.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
BACKGROUND Accurate rapid diagnostic tests for SARS-CoV-2 infection would be a useful tool to help manage the COVID-19 pandemic. Testing strategies that use rapid antigen tests to detect current infection have the potential to increase access to testing, speed detection of infection, and inform clinical and public health management decisions to reduce transmission. This is the second update of this review, which was first published in 2020. OBJECTIVES To assess the diagnostic accuracy of rapid, point-of-care antigen tests for diagnosis of SARS-CoV-2 infection. We consider accuracy separately in symptomatic and asymptomatic population groups. Sources of heterogeneity investigated included setting and indication for testing, assay format, sample site, viral load, age, timing of test, and study design. SEARCH METHODS We searched the COVID-19 Open Access Project living evidence database from the University of Bern (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) on 08 March 2021. We included independent evaluations from national reference laboratories, FIND and the Diagnostics Global Health website. We did not apply language restrictions. SELECTION CRITERIA We included studies of people with either suspected SARS-CoV-2 infection, known SARS-CoV-2 infection or known absence of infection, or those who were being screened for infection. We included test accuracy studies of any design that evaluated commercially produced, rapid antigen tests. We included evaluations of single applications of a test (one test result reported per person) and evaluations of serial testing (repeated antigen testing over time). Reference standards for presence or absence of infection were any laboratory-based molecular test (primarily reverse transcription polymerase chain reaction (RT-PCR)) or pre-pandemic respiratory sample. DATA COLLECTION AND ANALYSIS We used standard screening procedures with three people. Two people independently carried out quality assessment (using the QUADAS-2 tool) and extracted study results. Other study characteristics were extracted by one review author and checked by a second. We present sensitivity and specificity with 95% confidence intervals (CIs) for each test, and pooled data using the bivariate model. We investigated heterogeneity by including indicator variables in the random-effects logistic regression models. We tabulated results by test manufacturer and compliance with manufacturer instructions for use and according to symptom status. MAIN RESULTS We included 155 study cohorts (described in 166 study reports, with 24 as preprints). The main results relate to 152 evaluations of single test applications including 100,462 unique samples (16,822 with confirmed SARS-CoV-2). Studies were mainly conducted in Europe (101/152, 66%), and evaluated 49 different commercial antigen assays. Only 23 studies compared two or more brands of test. Risk of bias was high because of participant selection (40, 26%); interpretation of the index test (6, 4%); weaknesses in the reference standard for absence of infection (119, 78%); and participant flow and timing 41 (27%). Characteristics of participants (45, 30%) and index test delivery (47, 31%) differed from the way in which and in whom the test was intended to be used. Nearly all studies (91%) used a single RT-PCR result to define presence or absence of infection. The 152 studies of single test applications reported 228 evaluations of antigen tests. Estimates of sensitivity varied considerably between studies, with consistently high specificities. Average sensitivity was higher in symptomatic (73.0%, 95% CI 69.3% to 76.4%; 109 evaluations; 50,574 samples, 11,662 cases) compared to asymptomatic participants (54.7%, 95% CI 47.7% to 61.6%; 50 evaluations; 40,956 samples, 2641 cases). Average sensitivity was higher in the first week after symptom onset (80.9%, 95% CI 76.9% to 84.4%; 30 evaluations, 2408 cases) than in the second week of symptoms (53.8%, 95% CI 48.0% to 59.6%; 40 evaluations, 1119 cases). For those who were asymptomatic at the time of testing, sensitivity was higher when an epidemiological exposure to SARS-CoV-2 was suspected (64.3%, 95% CI 54.6% to 73.0%; 16 evaluations; 7677 samples, 703 cases) compared to where COVID-19 testing was reported to be widely available to anyone on presentation for testing (49.6%, 95% CI 42.1% to 57.1%; 26 evaluations; 31,904 samples, 1758 cases). Average specificity was similarly high for symptomatic (99.1%) or asymptomatic (99.7%) participants. We observed a steady decline in summary sensitivities as measures of sample viral load decreased. Sensitivity varied between brands. When tests were used according to manufacturer instructions, average sensitivities by brand ranged from 34.3% to 91.3% in symptomatic participants (20 assays with eligible data) and from 28.6% to 77.8% for asymptomatic participants (12 assays). For symptomatic participants, summary sensitivities for seven assays were 80% or more (meeting acceptable criteria set by the World Health Organization (WHO)). The WHO acceptable performance criterion of 97% specificity was met by 17 of 20 assays when tests were used according to manufacturer instructions, 12 of which demonstrated specificities above 99%. For asymptomatic participants the sensitivities of only two assays approached but did not meet WHO acceptable performance standards in one study each; specificities for asymptomatic participants were in a similar range to those observed for symptomatic people. At 5% prevalence using summary data in symptomatic people during the first week after symptom onset, the positive predictive value (PPV) of 89% means that 1 in 10 positive results will be a false positive, and around 1 in 5 cases will be missed. At 0.5% prevalence using summary data for asymptomatic people, where testing was widely available and where epidemiological exposure to COVID-19 was suspected, resulting PPVs would be 38% to 52%, meaning that between 2 in 5 and 1 in 2 positive results will be false positives, and between 1 in 2 and 1 in 3 cases will be missed. AUTHORS' CONCLUSIONS Antigen tests vary in sensitivity. In people with signs and symptoms of COVID-19, sensitivities are highest in the first week of illness when viral loads are higher. Assays that meet appropriate performance standards, such as those set by WHO, could replace laboratory-based RT-PCR when immediate decisions about patient care must be made, or where RT-PCR cannot be delivered in a timely manner. However, they are more suitable for use as triage to RT-PCR testing. The variable sensitivity of antigen tests means that people who test negative may still be infected. Many commercially available rapid antigen tests have not been evaluated in independent validation studies. Evidence for testing in asymptomatic cohorts has increased, however sensitivity is lower and there is a paucity of evidence for testing in different settings. Questions remain about the use of antigen test-based repeat testing strategies. Further research is needed to evaluate the effectiveness of screening programmes at reducing transmission of infection, whether mass screening or targeted approaches including schools, healthcare setting and traveller screening.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Pawana Sharma
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Sarah Berhane
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Susanna S van Wyk
- Centre for Evidence-based Health Care, Epidemiology and Biostatistics, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa
| | - Nicholas Nyaaba
- Infectious Disease Unit, 37 Military Hospital, Cantonments, Ghana
| | - Julie Domen
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Melissa Taylor
- Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool, UK
| | - Jane Cunningham
- Global Malaria Programme, World Health Organization, Geneva, Switzerland
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | | | | | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | | | - René Spijker
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
| | - Jan Y Verbakel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Sian Taylor-Phillips
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| |
Collapse
|
11
|
Struyf T, Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Leeflang MM, Spijker R, Hooft L, Emperador D, Domen J, Tans A, Janssens S, Wickramasinghe D, Lannoy V, Horn SRA, Van den Bruel A. Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has COVID-19. Cochrane Database Syst Rev 2022; 5:CD013665. [PMID: 35593186 PMCID: PMC9121352 DOI: 10.1002/14651858.cd013665.pub3] [Citation(s) in RCA: 36] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/26/2023]
Abstract
BACKGROUND COVID-19 illness is highly variable, ranging from infection with no symptoms through to pneumonia and life-threatening consequences. Symptoms such as fever, cough, or loss of sense of smell (anosmia) or taste (ageusia), can help flag early on if the disease is present. Such information could be used either to rule out COVID-19 disease, or to identify people who need to go for COVID-19 diagnostic tests. This is the second update of this review, which was first published in 2020. OBJECTIVES To assess the diagnostic accuracy of signs and symptoms to determine if a person presenting in primary care or to hospital outpatient settings, such as the emergency department or dedicated COVID-19 clinics, has COVID-19. SEARCH METHODS We undertook electronic searches up to 10 June 2021 in the University of Bern living search database. In addition, we checked repositories of COVID-19 publications. We used artificial intelligence text analysis to conduct an initial classification of documents. We did not apply any language restrictions. SELECTION CRITERIA Studies were eligible if they included people with clinically suspected COVID-19, or recruited known cases with COVID-19 and also controls without COVID-19 from a single-gate cohort. Studies were eligible when they recruited people presenting to primary care or hospital outpatient settings. Studies that included people who contracted SARS-CoV-2 infection while admitted to hospital were not eligible. The minimum eligible sample size of studies was 10 participants. All signs and symptoms were eligible for this review, including individual signs and symptoms or combinations. We accepted a range of reference standards. DATA COLLECTION AND ANALYSIS Pairs of review authors independently selected all studies, at both title and abstract, and full-text stage. They resolved any disagreements by discussion with a third review author. Two review authors independently extracted data and assessed risk of bias using the QUADAS-2 checklist, and resolved disagreements by discussion with a third review author. Analyses were restricted to prospective studies only. We presented sensitivity and specificity in paired forest plots, in receiver operating characteristic (ROC) space and in dumbbell plots. We estimated summary parameters using a bivariate random-effects meta-analysis whenever five or more primary prospective studies were available, and whenever heterogeneity across studies was deemed acceptable. MAIN RESULTS We identified 90 studies; for this update we focused on the results of 42 prospective studies with 52,608 participants. Prevalence of COVID-19 disease varied from 3.7% to 60.6% with a median of 27.4%. Thirty-five studies were set in emergency departments or outpatient test centres (46,878 participants), three in primary care settings (1230 participants), two in a mixed population of in- and outpatients in a paediatric hospital setting (493 participants), and two overlapping studies in nursing homes (4007 participants). The studies did not clearly distinguish mild COVID-19 disease from COVID-19 pneumonia, so we present the results for both conditions together. Twelve studies had a high risk of bias for selection of participants because they used a high level of preselection to decide whether reverse transcription polymerase chain reaction (RT-PCR) testing was needed, or because they enrolled a non-consecutive sample, or because they excluded individuals while they were part of the study base. We rated 36 of the 42 studies as high risk of bias for the index tests because there was little or no detail on how, by whom and when, the symptoms were measured. For most studies, eligibility for testing was dependent on the local case definition and testing criteria that were in effect at the time of the study, meaning most people who were included in studies had already been referred to health services based on the symptoms that we are evaluating in this review. The applicability of the results of this review iteration improved in comparison with the previous reviews. This version has more studies of people presenting to ambulatory settings, which is where the majority of assessments for COVID-19 take place. Only three studies presented any data on children separately, and only one focused specifically on older adults. We found data on 96 symptoms or combinations of signs and symptoms. Evidence on individual signs as diagnostic tests was rarely reported, so this review reports mainly on the diagnostic value of symptoms. Results were highly variable across studies. Most had very low sensitivity and high specificity. RT-PCR was the most often used reference standard (40/42 studies). Only cough (11 studies) had a summary sensitivity above 50% (62.4%, 95% CI 50.6% to 72.9%)); its specificity was low (45.4%, 95% CI 33.5% to 57.9%)). Presence of fever had a sensitivity of 37.6% (95% CI 23.4% to 54.3%) and a specificity of 75.2% (95% CI 56.3% to 87.8%). The summary positive likelihood ratio of cough was 1.14 (95% CI 1.04 to 1.25) and that of fever 1.52 (95% CI 1.10 to 2.10). Sore throat had a summary positive likelihood ratio of 0.814 (95% CI 0.714 to 0.929), which means that its presence increases the probability of having an infectious disease other than COVID-19. Dyspnoea (12 studies) and fatigue (8 studies) had a sensitivity of 23.3% (95% CI 16.4% to 31.9%) and 40.2% (95% CI 19.4% to 65.1%) respectively. Their specificity was 75.7% (95% CI 65.2% to 83.9%) and 73.6% (95% CI 48.4% to 89.3%). The summary positive likelihood ratio of dyspnoea was 0.96 (95% CI 0.83 to 1.11) and that of fatigue 1.52 (95% CI 1.21 to 1.91), which means that the presence of fatigue slightly increases the probability of having COVID-19. Anosmia alone (7 studies), ageusia alone (5 studies), and anosmia or ageusia (6 studies) had summary sensitivities below 50% but summary specificities over 90%. Anosmia had a summary sensitivity of 26.4% (95% CI 13.8% to 44.6%) and a specificity of 94.2% (95% CI 90.6% to 96.5%). Ageusia had a summary sensitivity of 23.2% (95% CI 10.6% to 43.3%) and a specificity of 92.6% (95% CI 83.1% to 97.0%). Anosmia or ageusia had a summary sensitivity of 39.2% (95% CI 26.5% to 53.6%) and a specificity of 92.1% (95% CI 84.5% to 96.2%). The summary positive likelihood ratios of anosmia alone and anosmia or ageusia were 4.55 (95% CI 3.46 to 5.97) and 4.99 (95% CI 3.22 to 7.75) respectively, which is just below our arbitrary definition of a 'red flag', that is, a positive likelihood ratio of at least 5. The summary positive likelihood ratio of ageusia alone was 3.14 (95% CI 1.79 to 5.51). Twenty-four studies assessed combinations of different signs and symptoms, mostly combining olfactory symptoms. By combining symptoms with other information such as contact or travel history, age, gender, and a local recent case detection rate, some multivariable prediction scores reached a sensitivity as high as 90%. AUTHORS' CONCLUSIONS Most individual symptoms included in this review have poor diagnostic accuracy. Neither absence nor presence of symptoms are accurate enough to rule in or rule out the disease. The presence of anosmia or ageusia may be useful as a red flag for the presence of COVID-19. The presence of cough also supports further testing. There is currently no evidence to support further testing with PCR in any individuals presenting only with upper respiratory symptoms such as sore throat, coryza or rhinorrhoea. Combinations of symptoms with other readily available information such as contact or travel history, or the local recent case detection rate may prove more useful and should be further investigated in an unselected population presenting to primary care or hospital outpatient settings. The diagnostic accuracy of symptoms for COVID-19 is moderate to low and any testing strategy using symptoms as selection mechanism will result in both large numbers of missed cases and large numbers of people requiring testing. Which one of these is minimised, is determined by the goal of COVID-19 testing strategies, that is, controlling the epidemic by isolating every possible case versus identifying those with clinically important disease so that they can be monitored or treated to optimise their prognosis. The former will require a testing strategy that uses very few symptoms as entry criterion for testing, the latter could focus on more specific symptoms such as fever and anosmia.
Collapse
Affiliation(s)
- Thomas Struyf
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | - René Spijker
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | | | - Julie Domen
- Department of Primary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium
| | - Anouk Tans
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | | | | | | | - Sebastiaan R A Horn
- Department of Primary Care, Faculty of Medicine and Health Sciences, University of Antwerp, Antwerp, Belgium
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| |
Collapse
|
12
|
Ebrahimzadeh S, Islam N, Dawit H, Salameh JP, Kazi S, Fabiano N, Treanor L, Absi M, Ahmad F, Rooprai P, Al Khalil A, Harper K, Kamra N, Leeflang MM, Hooft L, van der Pol CB, Prager R, Hare SS, Dennie C, Spijker R, Deeks JJ, Dinnes J, Jenniskens K, Korevaar DA, Cohen JF, Van den Bruel A, Takwoingi Y, van de Wijgert J, Wang J, Pena E, Sabongui S, McInnes MD. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2022; 5:CD013639. [PMID: 35575286 PMCID: PMC9109458 DOI: 10.1002/14651858.cd013639.pub5] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
BACKGROUND Our March 2021 edition of this review showed thoracic imaging computed tomography (CT) to be sensitive and moderately specific in diagnosing COVID-19 pneumonia. This new edition is an update of the review. OBJECTIVES Our objectives were to evaluate the diagnostic accuracy of thoracic imaging in people with suspected COVID-19; assess the rate of positive imaging in people who had an initial reverse transcriptase polymerase chain reaction (RT-PCR) negative result and a positive RT-PCR result on follow-up; and evaluate the accuracy of thoracic imaging for screening COVID-19 in asymptomatic individuals. The secondary objective was to assess threshold effects of index test positivity on accuracy. SEARCH METHODS We searched the COVID-19 Living Evidence Database from the University of Bern, the Cochrane COVID-19 Study Register, The Stephen B. Thacker CDC Library, and repositories of COVID-19 publications through to 17 February 2021. We did not apply any language restrictions. SELECTION CRITERIA We included diagnostic accuracy studies of all designs, except for case-control, that recruited participants of any age group suspected to have COVID-19. Studies had to assess chest CT, chest X-ray, or ultrasound of the lungs for the diagnosis of COVID-19, use a reference standard that included RT-PCR, and report estimates of test accuracy or provide data from which we could compute estimates. We excluded studies that used imaging as part of the reference standard and studies that excluded participants with normal index test results. DATA COLLECTION AND ANALYSIS The review authors independently and in duplicate screened articles, extracted data and assessed risk of bias and applicability concerns using QUADAS-2. We presented sensitivity and specificity per study on paired forest plots, and summarized pooled estimates in tables. We used a bivariate meta-analysis model where appropriate. MAIN RESULTS We included 98 studies in this review. Of these, 94 were included for evaluating the diagnostic accuracy of thoracic imaging in the evaluation of people with suspected COVID-19. Eight studies were included for assessing the rate of positive imaging in individuals with initial RT-PCR negative results and positive RT-PCR results on follow-up, and 10 studies were included for evaluating the accuracy of thoracic imaging for imagining asymptomatic individuals. For all 98 included studies, risk of bias was high or unclear in 52 (53%) studies with respect to participant selection, in 64 (65%) studies with respect to reference standard, in 46 (47%) studies with respect to index test, and in 48 (49%) studies with respect to flow and timing. Concerns about the applicability of the evidence to: participants were high or unclear in eight (8%) studies; index test were high or unclear in seven (7%) studies; and reference standard were high or unclear in seven (7%) studies. Imaging in people with suspected COVID-19 We included 94 studies. Eighty-seven studies evaluated one imaging modality, and seven studies evaluated two imaging modalities. All studies used RT-PCR alone or in combination with other criteria (for example, clinical signs and symptoms, positive contacts) as the reference standard for the diagnosis of COVID-19. For chest CT (69 studies, 28285 participants, 14,342 (51%) cases), sensitivities ranged from 45% to 100%, and specificities from 10% to 99%. The pooled sensitivity of chest CT was 86.9% (95% confidence interval (CI) 83.6 to 89.6), and pooled specificity was 78.3% (95% CI 73.7 to 82.3). Definition for index test positivity was a source of heterogeneity for sensitivity, but not specificity. Reference standard was not a source of heterogeneity. For chest X-ray (17 studies, 8529 participants, 5303 (62%) cases), the sensitivity ranged from 44% to 94% and specificity from 24 to 93%. The pooled sensitivity of chest X-ray was 73.1% (95% CI 64. to -80.5), and pooled specificity was 73.3% (95% CI 61.9 to 82.2). Definition for index test positivity was not found to be a source of heterogeneity. Definition for index test positivity and reference standard were not found to be sources of heterogeneity. For ultrasound of the lungs (15 studies, 2410 participants, 1158 (48%) cases), the sensitivity ranged from 73% to 94% and the specificity ranged from 21% to 98%. The pooled sensitivity of ultrasound was 88.9% (95% CI 84.9 to 92.0), and the pooled specificity was 72.2% (95% CI 58.8 to 82.5). Definition for index test positivity and reference standard were not found to be sources of heterogeneity. Indirect comparisons of modalities evaluated across all 94 studies indicated that chest CT and ultrasound gave higher sensitivity estimates than X-ray (P = 0.0003 and P = 0.001, respectively). Chest CT and ultrasound gave similar sensitivities (P=0.42). All modalities had similar specificities (CT versus X-ray P = 0.36; CT versus ultrasound P = 0.32; X-ray versus ultrasound P = 0.89). Imaging in PCR-negative people who subsequently became positive For rate of positive imaging in individuals with initial RT-PCR negative results, we included 8 studies (7 CT, 1 ultrasound) with a total of 198 participants suspected of having COVID-19, all of whom had a final diagnosis of COVID-19. Most studies (7/8) evaluated CT. Of 177 participants with initially negative RT-PCR who had positive RT-PCR results on follow-up testing, 75.8% (95% CI 45.3 to 92.2) had positive CT findings. Imaging in asymptomatic PCR-positive people For imaging asymptomatic individuals, we included 10 studies (7 CT, 1 X-ray, 2 ultrasound) with a total of 3548 asymptomatic participants, of whom 364 (10%) had a final diagnosis of COVID-19. For chest CT (7 studies, 3134 participants, 315 (10%) cases), the pooled sensitivity was 55.7% (95% CI 35.4 to 74.3) and the pooled specificity was 91.1% (95% CI 82.6 to 95.7). AUTHORS' CONCLUSIONS Chest CT and ultrasound of the lungs are sensitive and moderately specific in diagnosing COVID-19. Chest X-ray is moderately sensitive and moderately specific in diagnosing COVID-19. Thus, chest CT and ultrasound may have more utility for ruling out COVID-19 than for differentiating SARS-CoV-2 infection from other causes of respiratory illness. The uncertainty resulting from high or unclear risk of bias and the heterogeneity of included studies limit our ability to confidently draw conclusions based on our results.
Collapse
Affiliation(s)
- Sanam Ebrahimzadeh
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | - Nayaar Islam
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Haben Dawit
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | | | - Sakib Kazi
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | | | - Lee Treanor
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Marissa Absi
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Faraz Ahmad
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Paul Rooprai
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Ahmed Al Khalil
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Kelly Harper
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Neil Kamra
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht , Netherlands
| | | | - Ross Prager
- Department of Medicine, University of Ottawa, Ottawa, Canada
| | - Samanjit S Hare
- Department of Radiology, Royal Free London NHS Trust, London , UK
| | - Carole Dennie
- Department of Radiology, University of Ottawa, Ottawa, Canada
- Department of Medical Imaging, The Ottawa Hospital, Ottawa, Canada
| | - René Spijker
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht , Netherlands
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Kevin Jenniskens
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Daniël A Korevaar
- Department of Respiratory Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - Jérémie F Cohen
- Obstetrical, Perinatal and Pediatric Epidemiology Research Team (EPOPé), Centre of Research in Epidemiology and Statistics (CRESS), UMR1153, Université de Paris, Paris, France
| | | | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Janneke van de Wijgert
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Institute of Infection, Veterinary, and Ecological Sciences, University of Liverpool, Liverpool, UK
| | - Junfeng Wang
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, Netherlands
| | - Elena Pena
- Department of Radiology, University of Ottawa, Ottawa, Canada
- Department of Medical Imaging, The Ottawa Hospital, Ottawa, Canada
| | | | - Matthew Df McInnes
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- Department of Radiology, University of Ottawa, Ottawa, Canada
| |
Collapse
|
13
|
Yang B, Mallett S, Takwoingi Y, Davenport CF, Hyde CJ, Whiting PF, Deeks JJ, Leeflang MMG, Bossuyt PMM, Brazzelli MG, Dinnes J, Gurusamy KS, Jones HE, Lange S, Langendam MW, Macaskill P, McInnes MDF, Reitsma JB, Rutjes AWS, Sinclair A, de Vet HCW, Virgili G, Wade R, Westwood ME. QUADAS-C: A Tool for Assessing Risk of Bias in Comparative Diagnostic Accuracy Studies. Ann Intern Med 2021; 174:1592-1599. [PMID: 34698503 DOI: 10.7326/m21-2234] [Citation(s) in RCA: 80] [Impact Index Per Article: 26.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Comparative diagnostic test accuracy studies assess and compare the accuracy of 2 or more tests in the same study. Although these studies have the potential to yield reliable evidence regarding comparative accuracy, shortcomings in the design, conduct, and analysis may bias their results. The currently recommended quality assessment tool for diagnostic test accuracy studies, QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies-2), is not designed for the assessment of test comparisons. The QUADAS-C (Quality Assessment of Diagnostic Accuracy Studies-Comparative) tool was developed as an extension of QUADAS-2 to assess the risk of bias in comparative diagnostic test accuracy studies. Through a 4-round Delphi study involving 24 international experts in test evaluation and a face-to-face consensus meeting, an initial version of the tool was developed that was revised and finalized following a pilot study among potential users. The QUADAS-C tool retains the same 4-domain structure of QUADAS-2 (Patient Selection, Index Test, Reference Standard, and Flow and Timing) and comprises additional questions to each QUADAS-2 domain. A risk-of-bias judgment for comparative accuracy requires a risk-of-bias judgment for the accuracy of each test (resulting from QUADAS-2) and additional criteria specific to test comparisons. Examples of such additional criteria include whether participants either received all index tests or were randomly assigned to index tests, and whether index tests were interpreted with blinding to the results of other index tests. The QUADAS-C tool will be useful for systematic reviews of diagnostic test accuracy addressing comparative questions. Furthermore, researchers may use this tool to identify and avoid risk of bias when designing a comparative diagnostic test accuracy study.
Collapse
Affiliation(s)
- Bada Yang
- Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands (B.Y., M.M.L.)
| | - Sue Mallett
- UCL Centre for Medical Imaging, University College London, London, United Kingdom (S.M.)
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, and National Institute for Health Research Birmingham Biomedical Research Centre, University Hospitals Birmingham National Health Service Foundation Trust and University of Birmingham, Birmingham, United Kingdom (Y.T., C.F.D., J.J.D.)
| | - Clare F Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, and National Institute for Health Research Birmingham Biomedical Research Centre, University Hospitals Birmingham National Health Service Foundation Trust and University of Birmingham, Birmingham, United Kingdom (Y.T., C.F.D., J.J.D.)
| | - Christopher J Hyde
- Exeter Test Group, The Institute of Health Research, College of Medicine and Health, University of Exeter, Exeter, United Kingdom (C.J.H.)
| | - Penny F Whiting
- Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, United Kingdom (P.F.W.)
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, and National Institute for Health Research Birmingham Biomedical Research Centre, University Hospitals Birmingham National Health Service Foundation Trust and University of Birmingham, Birmingham, United Kingdom (Y.T., C.F.D., J.J.D.)
| | - Mariska M G Leeflang
- Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, the Netherlands (B.Y., M.M.L.)
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
14
|
de Pablo P, Dinnes J, Berhane S, Osman A, Lim Z, Coombe A, Raza K, Filer A, Deeks JJ. Systematic review of imaging tests to predict the development of rheumatoid arthritis in people with unclassified arthritis. Semin Arthritis Rheum 2021; 52:151919. [PMID: 34782180 DOI: 10.1016/j.semarthrit.2021.10.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2021] [Revised: 09/16/2021] [Accepted: 10/18/2021] [Indexed: 11/19/2022]
Abstract
OBJECTIVE To estimate and compare the diagnostic accuracy of magnetic resonance imaging (MRI) and ultrasound, for the prediction of rheumatoid arthritis (RA) in unclassified arthritis (UA). METHODS MEDLINE, Embase and BIOSIS were searched from 1987 to May 2019. Studies evaluating any imaging test in participants with UA were eligible. Reference standards were RA classification criteria or methotrexate initiation. Two authors independently extracted data and assessed validity using QUADAS-2. Sensitivities and specificities were calculated for each imaging characteristic and joint area. Summary estimates with 95% confidence intervals (CI) were estimated where possible. RESULTS Nineteen studies were included; 13 evaluated MRI (n=1,143; 454 with RA) and 6 evaluated ultrasound (n=531; 205 with RA). Studies were limited by unclear recruitment procedures, inclusion of patients with RA at baseline, differential verification, lack of blinding and consensus grading. Study heterogeneity largely precluded meta-analysis, however summary sensitivity and specificity for MRI synovitis in at least one joint were 93% (95% CI 88%, 96%) and 25% (95% CI 13%, 41%) (3 studies). Specificities may be higher for other MRI characteristics but data are limited. Ultrasound results were difficult to synthesise due to different diagnostic thresholds and reference standards. CONCLUSION The evidence for MRI or ultrasound as single tests for predicting RA in people with UA is heterogeneous and of variable methodological quality. Larger studies using consensus grading and consistently defined RA diagnosis are needed to identify whether combinations of imaging characteristics, either alone or in combination with other clinical findings, can better predict RA in this population.
Collapse
Affiliation(s)
- Paola de Pablo
- Institute of Inflammation and Ageing, College of Medical & Dental Sciences, University of Birmingham, Birmingham, UK; Sandwell and West Birmingham Hospitals NHS Trust, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK; Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
| | - Sarah Berhane
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK; Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Aya Osman
- Institute of Inflammation and Ageing, College of Medical & Dental Sciences, University of Birmingham, Birmingham, UK
| | - Zhia Lim
- Institute of Inflammation and Ageing, College of Medical & Dental Sciences, University of Birmingham, Birmingham, UK
| | - April Coombe
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Karim Raza
- Institute of Inflammation and Ageing, College of Medical & Dental Sciences, University of Birmingham, Birmingham, UK; Sandwell and West Birmingham Hospitals NHS Trust, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Andrew Filer
- Institute of Inflammation and Ageing, College of Medical & Dental Sciences, University of Birmingham, Birmingham, UK; NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jonathan J Deeks
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK; Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| |
Collapse
|
15
|
Verbakel JY, De Rop L, Stegeman I, Holtman GA, Ochodo EA, Yang B, Guleid F, Davenport C, Deeks JJ, Dinnes J, Dittrich S, Emperador D, Hooft L, Spijker R, Van den Bruel A, Wang J, Takwoingi Y, Langendam MW, Leeflang MMG. Accuracy of routine laboratory tests to predict mortality and deterioration to severe or critical COVID-19 in people with SARS-CoV-2. Hippokratia 2021. [DOI: 10.1002/14651858.cd015050] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Affiliation(s)
- Jan Y Verbakel
- Department of Public Health and Primary Care; KU Leuven; Leuven Belgium
- Nuffield Department of Primary Care Health Sciences; University of Oxford; Oxford United Kingdom
| | - Liselore De Rop
- Department of Public Health and Primary Care; KU Leuven; Leuven Belgium
| | - Inge Stegeman
- Department of Otorhinolaryngology & Head and Neck Surgery; University Medical Center Utrecht; Utrecht Netherlands
- Brain Center Rudolf Magnus; University Medical Center Utrecht; Utrecht Netherlands
- Epidemiology and Data Science; Amsterdam University Medical Centers, University of Amsterdam; Amsterdam Netherlands
| | - Gea A. Holtman
- Department of General Practice; University of Groningen, University Medical Centre Groningen; Groningen Netherlands
| | - Eleanor A Ochodo
- Centre for Evidence-based Health Care, Department of Global Health, Faculty of Medicine and Health Sciences; Stellenbosch University; Cape Town South Africa
- Centre for Global Health Research; Kenya Medical Research Institute; Kisumu Kenya
| | - Bada Yang
- Epidemiology and Data Science; Amsterdam University Medical Centers, University of Amsterdam; Amsterdam Netherlands
| | - Fatuma Guleid
- KEMRI-Wellcome Trust Research Programme; Nairobi Kenya
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | | | | | - Lotty Hooft
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
| | - René Spijker
- Medical Library; Amsterdam UMC, University of Amsterdam, Amsterdam Public Health; Amsterdam Netherlands
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
| | - Ann Van den Bruel
- Department of Public Health and Primary Care; KU Leuven; Leuven Belgium
| | - Junfeng Wang
- Julius Center for Health Sciences and Primary Care; University Medical Center Utrecht; Utrecht Netherlands
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Miranda W Langendam
- Epidemiology and Data Science; Amsterdam University Medical Centers, University of Amsterdam; Amsterdam Netherlands
| | - Mariska MG Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics; Amsterdam University Medical Centers, University of Amsterdam; Amsterdam Netherlands
| | | |
Collapse
|
16
|
Ji-Xu A, Dinnes J, Matin RN. Establishing the use of total body photography among U.K. dermatologists. Clin Exp Dermatol 2021; 47:182-184. [PMID: 34382263 DOI: 10.1111/ced.14882] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Revised: 07/07/2021] [Accepted: 08/09/2021] [Indexed: 01/05/2023]
Abstract
Total body photography (TBP) is increasingly used to monitor skin lesions in individuals at high risk of melanoma. Two surveys of U.S. dermatologists demonstrated variable reasons for use and conflicting beliefs regarding TBP efficacy. Although TBP may reduce number-needed-to-biopsy when diagnosing melanoma, there is scarce evidence for TBP use in lower-risk populations and a lack of consensus on optimal TBP use. Thus, we undertook the first U.K. survey to assess TBP rationale, target populations, and protocols.
Collapse
Affiliation(s)
- A Ji-Xu
- Department of Dermatology, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom
| | - J Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, United Kingdom.,NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, United Kingdom
| | - R N Matin
- Department of Dermatology, Oxford University Hospitals NHS Foundation Trust, Oxford, United Kingdom
| |
Collapse
|
17
|
Deeks JJ, Dinnes J, Davenport C, Takwoingi Y, McInnes M, Leeflang MMG, Cunningham J. Letter to the Editor regarding Peto T; UK COVID-19 Lateral Flow Oversight Team: COVID-19: Rapid antigen detection for SARS-CoV-2 by lateral flow assay. EClinicalMedicine 2021; 38:101037. [PMID: 34308323 PMCID: PMC8280129 DOI: 10.1016/j.eclinm.2021.101037] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Accepted: 07/06/2021] [Indexed: 12/03/2022] Open
Affiliation(s)
- Jonathan J Deeks
- Institute of Applied Health Research, University of Birmingham, Birmingham B15 2TT, UK
- Corresponding author.
| | - Jacqueline Dinnes
- Institute of Applied Health Research, University of Birmingham, Birmingham B15 2TT, UK
| | - Clare Davenport
- Institute of Applied Health Research, University of Birmingham, Birmingham B15 2TT, UK
| | - Yemisi Takwoingi
- Institute of Applied Health Research, University of Birmingham, Birmingham B15 2TT, UK
| | | | | | | |
Collapse
|
18
|
Affiliation(s)
- Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, UK.
| |
Collapse
|
19
|
Affiliation(s)
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of applied Health Research, University of Birmingham, Birmingham, UK
| |
Collapse
|
20
|
Matin RN, Dinnes J. AI-based smartphone apps for risk assessment of skin cancer need more evaluation and better regulation. Br J Cancer 2021; 124:1749-1750. [PMID: 33742148 PMCID: PMC8144419 DOI: 10.1038/s41416-021-01302-3] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2020] [Revised: 01/26/2021] [Accepted: 02/03/2021] [Indexed: 11/08/2022] Open
Abstract
Smartphone applications ("apps") with artificial intelligence (AI) algorithms are increasingly used in healthcare. Widespread adoption of these apps must be supported by a robust evidence-base and app manufacturers' claims appropriately regulated. Current CE marking assessment processes inadequately protect the public against the risks created by using smartphone diagnostic apps.
Collapse
Affiliation(s)
- Rubeta N Matin
- Department of Dermatology, Oxford University Hospitals NHS Foundation Trust, Oxford, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham, UK.
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK.
| |
Collapse
|
21
|
Doust JA, Bell KJL, Leeflang MMG, Dinnes J, Lord SJ, Mallett S, van de Wijgert JHHM, Sandberg S, Adeli K, Deeks JJ, Bossuyt PM, Horvath AR. Guidance for the design and reporting of studies evaluating the clinical performance of tests for present or past SARS-CoV-2 infection. BMJ 2021; 372:n568. [PMID: 33782084 DOI: 10.1136/bmj.n568] [Citation(s) in RCA: 12] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Affiliation(s)
- Jenny A Doust
- Centre for Longitudinal and Life Course Research, School of Public Health, University of Queensland, Herston, QLD 4006, Australia
| | - Katy J L Bell
- School of Public Health, University of Sydney, NSW, Australia
| | - Mariska M G Leeflang
- Department of Epidemiology and Data Science, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, Netherlands
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Sally J Lord
- School of Medicine, Sydney, University of Notre Dame, Darlinghurst, NSW, Australia
| | - Sue Mallett
- Centre for Medical Imaging, University College, London, UK
| | - Janneke H H M van de Wijgert
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
- Institute of Infection, Veterinary, and Ecological Sciences, University of Liverpool, Liverpool, UK
| | - Sverre Sandberg
- Department of Global Public Health and Primary Care, University of Bergen, Bergen, Norway
- Norwegian Quality Improvement of Laboratory Examinations, Haraldsplass Deaconess Hospital, Bergen, Norway
| | - Khosrow Adeli
- CALIPER Program, Paediatric Laboratory Medicine, The Hospital for Sick Children, Toronto, ON, Canada
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, ON, Canada
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Patrick M Bossuyt
- Department of Epidemiology and Data Science, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, Netherlands
| | - Andrea R Horvath
- School of Public Health, University of Sydney, NSW, Australia
- New South Wales Health Pathology, Department of Chemical Pathology, Prince of Wales Hospital, Sydney, NSW, Australia
- School of Medical Sciences, University of New South Wales, Sydney, NSW, Australia
| |
Collapse
|
22
|
Dinnes J, Deeks JJ, Berhane S, Taylor M, Adriano A, Davenport C, Dittrich S, Emperador D, Takwoingi Y, Cunningham J, Beese S, Domen J, Dretzke J, Ferrante di Ruffano L, Harris IM, Price MJ, Taylor-Phillips S, Hooft L, Leeflang MM, McInnes MD, Spijker R, Van den Bruel A. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev 2021; 3:CD013705. [PMID: 33760236 PMCID: PMC8078597 DOI: 10.1002/14651858.cd013705.pub2] [Citation(s) in RCA: 291] [Impact Index Per Article: 97.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Abstract
BACKGROUND Accurate rapid diagnostic tests for SARS-CoV-2 infection could contribute to clinical and public health strategies to manage the COVID-19 pandemic. Point-of-care antigen and molecular tests to detect current infection could increase access to testing and early confirmation of cases, and expediate clinical and public health management decisions that may reduce transmission. OBJECTIVES To assess the diagnostic accuracy of point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. We consider accuracy separately in symptomatic and asymptomatic population groups. SEARCH METHODS Electronic searches of the Cochrane COVID-19 Study Register and the COVID-19 Living Evidence Database from the University of Bern (which includes daily updates from PubMed and Embase and preprints from medRxiv and bioRxiv) were undertaken on 30 Sept 2020. We checked repositories of COVID-19 publications and included independent evaluations from national reference laboratories, the Foundation for Innovative New Diagnostics and the Diagnostics Global Health website to 16 Nov 2020. We did not apply language restrictions. SELECTION CRITERIA We included studies of people with either suspected SARS-CoV-2 infection, known SARS-CoV-2 infection or known absence of infection, or those who were being screened for infection. We included test accuracy studies of any design that evaluated commercially produced, rapid antigen or molecular tests suitable for a point-of-care setting (minimal equipment, sample preparation, and biosafety requirements, with results within two hours of sample collection). We included all reference standards that define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction (RT-PCR) tests and established diagnostic criteria). DATA COLLECTION AND ANALYSIS Studies were screened independently in duplicate with disagreements resolved by discussion with a third author. Study characteristics were extracted by one author and checked by a second; extraction of study results and assessments of risk of bias and applicability (made using the QUADAS-2 tool) were undertaken independently in duplicate. We present sensitivity and specificity with 95% confidence intervals (CIs) for each test and pooled data using the bivariate model separately for antigen and molecular-based tests. We tabulated results by test manufacturer and compliance with manufacturer instructions for use and according to symptom status. MAIN RESULTS Seventy-eight study cohorts were included (described in 64 study reports, including 20 pre-prints), reporting results for 24,087 samples (7,415 with confirmed SARS-CoV-2). Studies were mainly from Europe (n = 39) or North America (n = 20), and evaluated 16 antigen and five molecular assays. We considered risk of bias to be high in 29 (50%) studies because of participant selection; in 66 (85%) because of weaknesses in the reference standard for absence of infection; and in 29 (45%) for participant flow and timing. Studies of antigen tests were of a higher methodological quality compared to studies of molecular tests, particularly regarding the risk of bias for participant selection and the index test. Characteristics of participants in 35 (45%) studies differed from those in whom the test was intended to be used and the delivery of the index test in 39 (50%) studies differed from the way in which the test was intended to be used. Nearly all studies (97%) defined the presence or absence of SARS-CoV-2 based on a single RT-PCR result, and none included participants meeting case definitions for probable COVID-19. Antigen tests Forty-eight studies reported 58 evaluations of antigen tests. Estimates of sensitivity varied considerably between studies. There were differences between symptomatic (72.0%, 95% CI 63.7% to 79.0%; 37 evaluations; 15530 samples, 4410 cases) and asymptomatic participants (58.1%, 95% CI 40.2% to 74.1%; 12 evaluations; 1581 samples, 295 cases). Average sensitivity was higher in the first week after symptom onset (78.3%, 95% CI 71.1% to 84.1%; 26 evaluations; 5769 samples, 2320 cases) than in the second week of symptoms (51.0%, 95% CI 40.8% to 61.0%; 22 evaluations; 935 samples, 692 cases). Sensitivity was high in those with cycle threshold (Ct) values on PCR ≤25 (94.5%, 95% CI 91.0% to 96.7%; 36 evaluations; 2613 cases) compared to those with Ct values >25 (40.7%, 95% CI 31.8% to 50.3%; 36 evaluations; 2632 cases). Sensitivity varied between brands. Using data from instructions for use (IFU) compliant evaluations in symptomatic participants, summary sensitivities ranged from 34.1% (95% CI 29.7% to 38.8%; Coris Bioconcept) to 88.1% (95% CI 84.2% to 91.1%; SD Biosensor STANDARD Q). Average specificities were high in symptomatic and asymptomatic participants, and for most brands (overall summary specificity 99.6%, 95% CI 99.0% to 99.8%). At 5% prevalence using data for the most sensitive assays in symptomatic people (SD Biosensor STANDARD Q and Abbott Panbio), positive predictive values (PPVs) of 84% to 90% mean that between 1 in 10 and 1 in 6 positive results will be a false positive, and between 1 in 4 and 1 in 8 cases will be missed. At 0.5% prevalence applying the same tests in asymptomatic people would result in PPVs of 11% to 28% meaning that between 7 in 10 and 9 in 10 positive results will be false positives, and between 1 in 2 and 1 in 3 cases will be missed. No studies assessed the accuracy of repeated lateral flow testing or self-testing. Rapid molecular assays Thirty studies reported 33 evaluations of five different rapid molecular tests. Sensitivities varied according to test brand. Most of the data relate to the ID NOW and Xpert Xpress assays. Using data from evaluations following the manufacturer's instructions for use, the average sensitivity of ID NOW was 73.0% (95% CI 66.8% to 78.4%) and average specificity 99.7% (95% CI 98.7% to 99.9%; 4 evaluations; 812 samples, 222 cases). For Xpert Xpress, the average sensitivity was 100% (95% CI 88.1% to 100%) and average specificity 97.2% (95% CI 89.4% to 99.3%; 2 evaluations; 100 samples, 29 cases). Insufficient data were available to investigate the effect of symptom status or time after symptom onset. AUTHORS' CONCLUSIONS Antigen tests vary in sensitivity. In people with signs and symptoms of COVID-19, sensitivities are highest in the first week of illness when viral loads are higher. The assays shown to meet appropriate criteria, such as WHO's priority target product profiles for COVID-19 diagnostics ('acceptable' sensitivity ≥ 80% and specificity ≥ 97%), can be considered as a replacement for laboratory-based RT-PCR when immediate decisions about patient care must be made, or where RT-PCR cannot be delivered in a timely manner. Positive predictive values suggest that confirmatory testing of those with positive results may be considered in low prevalence settings. Due to the variable sensitivity of antigen tests, people who test negative may still be infected. Evidence for testing in asymptomatic cohorts was limited. Test accuracy studies cannot adequately assess the ability of antigen tests to differentiate those who are infectious and require isolation from those who pose no risk, as there is no reference standard for infectiousness. A small number of molecular tests showed high accuracy and may be suitable alternatives to RT-PCR. However, further evaluations of the tests in settings as they are intended to be used are required to fully establish performance in practice. Several important studies in asymptomatic individuals have been reported since the close of our search and will be incorporated at the next update of this review. Comparative studies of antigen tests in their intended use settings and according to test operator (including self-testing) are required.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham , UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jonathan J Deeks
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Sarah Berhane
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Melissa Taylor
- Department of Clinical Sciences, Liverpool School of Tropical Medicine, Liverpool, UK
| | - Ada Adriano
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Clare Davenport
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | | | | | - Yemisi Takwoingi
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Jane Cunningham
- Global Malaria Programme, World Health Organization, Geneva , Switzerland
| | - Sophie Beese
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Julie Domen
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Janine Dretzke
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Lavinia Ferrante di Ruffano
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Isobel M Harris
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Malcolm J Price
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Sian Taylor-Phillips
- Division of Health Sciences, Warwick Medical School, University of Warwick , Coventry, UK
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht , Netherlands
| | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | | | - René Spijker
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| |
Collapse
|
23
|
Islam N, Ebrahimzadeh S, Salameh JP, Kazi S, Fabiano N, Treanor L, Absi M, Hallgrimson Z, Leeflang MM, Hooft L, van der Pol CB, Prager R, Hare SS, Dennie C, Spijker R, Deeks JJ, Dinnes J, Jenniskens K, Korevaar DA, Cohen JF, Van den Bruel A, Takwoingi Y, van de Wijgert J, Damen JA, Wang J, McInnes MD. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2021; 3:CD013639. [PMID: 33724443 PMCID: PMC8078565 DOI: 10.1002/14651858.cd013639.pub4] [Citation(s) in RCA: 52] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
BACKGROUND The respiratory illness caused by SARS-CoV-2 infection continues to present diagnostic challenges. Our 2020 edition of this review showed thoracic (chest) imaging to be sensitive and moderately specific in the diagnosis of coronavirus disease 2019 (COVID-19). In this update, we include new relevant studies, and have removed studies with case-control designs, and those not intended to be diagnostic test accuracy studies. OBJECTIVES To evaluate the diagnostic accuracy of thoracic imaging (computed tomography (CT), X-ray and ultrasound) in people with suspected COVID-19. SEARCH METHODS We searched the COVID-19 Living Evidence Database from the University of Bern, the Cochrane COVID-19 Study Register, The Stephen B. Thacker CDC Library, and repositories of COVID-19 publications through to 30 September 2020. We did not apply any language restrictions. SELECTION CRITERIA We included studies of all designs, except for case-control, that recruited participants of any age group suspected to have COVID-19 and that reported estimates of test accuracy or provided data from which we could compute estimates. DATA COLLECTION AND ANALYSIS The review authors independently and in duplicate screened articles, extracted data and assessed risk of bias and applicability concerns using the QUADAS-2 domain-list. We presented the results of estimated sensitivity and specificity using paired forest plots, and we summarised pooled estimates in tables. We used a bivariate meta-analysis model where appropriate. We presented the uncertainty of accuracy estimates using 95% confidence intervals (CIs). MAIN RESULTS We included 51 studies with 19,775 participants suspected of having COVID-19, of whom 10,155 (51%) had a final diagnosis of COVID-19. Forty-seven studies evaluated one imaging modality each, and four studies evaluated two imaging modalities each. All studies used RT-PCR as the reference standard for the diagnosis of COVID-19, with 47 studies using only RT-PCR and four studies using a combination of RT-PCR and other criteria (such as clinical signs, imaging tests, positive contacts, and follow-up phone calls) as the reference standard. Studies were conducted in Europe (33), Asia (13), North America (3) and South America (2); including only adults (26), all ages (21), children only (1), adults over 70 years (1), and unclear (2); in inpatients (2), outpatients (32), and setting unclear (17). Risk of bias was high or unclear in thirty-two (63%) studies with respect to participant selection, 40 (78%) studies with respect to reference standard, 30 (59%) studies with respect to index test, and 24 (47%) studies with respect to participant flow. For chest CT (41 studies, 16,133 participants, 8110 (50%) cases), the sensitivity ranged from 56.3% to 100%, and specificity ranged from 25.4% to 97.4%. The pooled sensitivity of chest CT was 87.9% (95% CI 84.6 to 90.6) and the pooled specificity was 80.0% (95% CI 74.9 to 84.3). There was no statistical evidence indicating that reference standard conduct and definition for index test positivity were sources of heterogeneity for CT studies. Nine chest CT studies (2807 participants, 1139 (41%) cases) used the COVID-19 Reporting and Data System (CO-RADS) scoring system, which has five thresholds to define index test positivity. At a CO-RADS threshold of 5 (7 studies), the sensitivity ranged from 41.5% to 77.9% and the pooled sensitivity was 67.0% (95% CI 56.4 to 76.2); the specificity ranged from 83.5% to 96.2%; and the pooled specificity was 91.3% (95% CI 87.6 to 94.0). At a CO-RADS threshold of 4 (7 studies), the sensitivity ranged from 56.3% to 92.9% and the pooled sensitivity was 83.5% (95% CI 74.4 to 89.7); the specificity ranged from 77.2% to 90.4% and the pooled specificity was 83.6% (95% CI 80.5 to 86.4). For chest X-ray (9 studies, 3694 participants, 2111 (57%) cases) the sensitivity ranged from 51.9% to 94.4% and specificity ranged from 40.4% to 88.9%. The pooled sensitivity of chest X-ray was 80.6% (95% CI 69.1 to 88.6) and the pooled specificity was 71.5% (95% CI 59.8 to 80.8). For ultrasound of the lungs (5 studies, 446 participants, 211 (47%) cases) the sensitivity ranged from 68.2% to 96.8% and specificity ranged from 21.3% to 78.9%. The pooled sensitivity of ultrasound was 86.4% (95% CI 72.7 to 93.9) and the pooled specificity was 54.6% (95% CI 35.3 to 72.6). Based on an indirect comparison using all included studies, chest CT had a higher specificity than ultrasound. For indirect comparisons of chest CT and chest X-ray, or chest X-ray and ultrasound, the data did not show differences in specificity or sensitivity. AUTHORS' CONCLUSIONS Our findings indicate that chest CT is sensitive and moderately specific for the diagnosis of COVID-19. Chest X-ray is moderately sensitive and moderately specific for the diagnosis of COVID-19. Ultrasound is sensitive but not specific for the diagnosis of COVID-19. Thus, chest CT and ultrasound may have more utility for excluding COVID-19 than for differentiating SARS-CoV-2 infection from other causes of respiratory illness. Future diagnostic accuracy studies should pre-define positive imaging findings, include direct comparisons of the various modalities of interest in the same participant population, and implement improved reporting practices.
Collapse
Affiliation(s)
- Nayaar Islam
- Department of Radiology , University of Ottawa, Ottawa, Canada
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | | | | | - Sakib Kazi
- Department of Radiology , University of Ottawa, Ottawa, Canada
| | | | - Lee Treanor
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Marissa Absi
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | | | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht , Netherlands
| | | | - Ross Prager
- Department of Medicine, University of Ottawa , Ottawa, Canada
| | - Samanjit S Hare
- Department of Radiology , Royal Free London NHS Trust, London , UK
| | - Carole Dennie
- Department of Radiology , University of Ottawa, Ottawa, Canada
- Department of Medical Imaging, The Ottawa Hospital, Ottawa, Canada
| | - René Spijker
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht , Netherlands
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
| | - Jonathan J Deeks
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham , UK
| | - Kevin Jenniskens
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Daniël A Korevaar
- Department of Respiratory Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - Jérémie F Cohen
- Obstetrical, Perinatal and Pediatric Epidemiology Research Team (EPOPé), Centre of Research in Epidemiology and Statistics (CRESS), UMR1153, Université de Paris, Paris, France
| | | | - Yemisi Takwoingi
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Janneke van de Wijgert
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Institute of Infection, Veterinary, and Ecological Sciences, University of Liverpool, Liverpool, UK
| | - Johanna Aag Damen
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Junfeng Wang
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, Netherlands
| | - Matthew Df McInnes
- Department of Radiology, University of Ottawa, Ottawa, Canada
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| |
Collapse
|
24
|
Ji‐Xu A, Dinnes J, Matin R. Total body photography for the diagnosis of cutaneous melanoma in adults: a systematic review and meta‐analysis*. Br J Dermatol 2021; 185:302-312. [DOI: 10.1111/bjd.19759] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/24/2020] [Indexed: 01/10/2023]
Affiliation(s)
- A. Ji‐Xu
- Department of Dermatology Oxford University Hospitals NHS Foundation Trust Oxford UK
| | - J. Dinnes
- Test Evaluation Research Group Institute of Applied Health Research University of Birmingham Birmingham UK
- NIHR Birmingham Biomedical Research Centre University Hospitals Birmingham NHS Foundation Trust and University of Birmingham Birmingham UK
| | - R.N. Matin
- Department of Dermatology Oxford University Hospitals NHS Foundation Trust Oxford UK
| |
Collapse
|
25
|
Struyf T, Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Leeflang MM, Spijker R, Hooft L, Emperador D, Domen J, Horn SRA, Van den Bruel A. Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has COVID-19. Cochrane Database Syst Rev 2021; 2:CD013665. [PMID: 33620086 PMCID: PMC8407425 DOI: 10.1002/14651858.cd013665.pub2] [Citation(s) in RCA: 67] [Impact Index Per Article: 22.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
BACKGROUND The clinical implications of SARS-CoV-2 infection are highly variable. Some people with SARS-CoV-2 infection remain asymptomatic, whilst the infection can cause mild to moderate COVID-19 and COVID-19 pneumonia in others. This can lead to some people requiring intensive care support and, in some cases, to death, especially in older adults. Symptoms such as fever, cough, or loss of smell or taste, and signs such as oxygen saturation are the first and most readily available diagnostic information. Such information could be used to either rule out COVID-19, or select patients for further testing. This is an update of this review, the first version of which published in July 2020. OBJECTIVES To assess the diagnostic accuracy of signs and symptoms to determine if a person presenting in primary care or to hospital outpatient settings, such as the emergency department or dedicated COVID-19 clinics, has COVID-19. SEARCH METHODS For this review iteration we undertook electronic searches up to 15 July 2020 in the Cochrane COVID-19 Study Register and the University of Bern living search database. In addition, we checked repositories of COVID-19 publications. We did not apply any language restrictions. SELECTION CRITERIA Studies were eligible if they included patients with clinically suspected COVID-19, or if they recruited known cases with COVID-19 and controls without COVID-19. Studies were eligible when they recruited patients presenting to primary care or hospital outpatient settings. Studies in hospitalised patients were only included if symptoms and signs were recorded on admission or at presentation. Studies including patients who contracted SARS-CoV-2 infection while admitted to hospital were not eligible. The minimum eligible sample size of studies was 10 participants. All signs and symptoms were eligible for this review, including individual signs and symptoms or combinations. We accepted a range of reference standards. DATA COLLECTION AND ANALYSIS Pairs of review authors independently selected all studies, at both title and abstract stage and full-text stage. They resolved any disagreements by discussion with a third review author. Two review authors independently extracted data and resolved disagreements by discussion with a third review author. Two review authors independently assessed risk of bias using the Quality Assessment tool for Diagnostic Accuracy Studies (QUADAS-2) checklist. We presented sensitivity and specificity in paired forest plots, in receiver operating characteristic space and in dumbbell plots. We estimated summary parameters using a bivariate random-effects meta-analysis whenever five or more primary studies were available, and whenever heterogeneity across studies was deemed acceptable. MAIN RESULTS We identified 44 studies including 26,884 participants in total. Prevalence of COVID-19 varied from 3% to 71% with a median of 21%. There were three studies from primary care settings (1824 participants), nine studies from outpatient testing centres (10,717 participants), 12 studies performed in hospital outpatient wards (5061 participants), seven studies in hospitalised patients (1048 participants), 10 studies in the emergency department (3173 participants), and three studies in which the setting was not specified (5061 participants). The studies did not clearly distinguish mild from severe COVID-19, so we present the results for all disease severities together. Fifteen studies had a high risk of bias for selection of participants because inclusion in the studies depended on the applicable testing and referral protocols, which included many of the signs and symptoms under study in this review. This may have especially influenced the sensitivity of those features used in referral protocols, such as fever and cough. Five studies only included participants with pneumonia on imaging, suggesting that this is a highly selected population. In an additional 12 studies, we were unable to assess the risk for selection bias. This makes it very difficult to judge the validity of the diagnostic accuracy of the signs and symptoms from these included studies. The applicability of the results of this review update improved in comparison with the original review. A greater proportion of studies included participants who presented to outpatient settings, which is where the majority of clinical assessments for COVID-19 take place. However, still none of the studies presented any data on children separately, and only one focused specifically on older adults. We found data on 84 signs and symptoms. Results were highly variable across studies. Most had very low sensitivity and high specificity. Only cough (25 studies) and fever (7 studies) had a pooled sensitivity of at least 50% but specificities were moderate to low. Cough had a sensitivity of 67.4% (95% confidence interval (CI) 59.8% to 74.1%) and specificity of 35.0% (95% CI 28.7% to 41.9%). Fever had a sensitivity of 53.8% (95% CI 35.0% to 71.7%) and a specificity of 67.4% (95% CI 53.3% to 78.9%). The pooled positive likelihood ratio of cough was only 1.04 (95% CI 0.97 to 1.11) and that of fever 1.65 (95% CI 1.41 to 1.93). Anosmia alone (11 studies), ageusia alone (6 studies), and anosmia or ageusia (6 studies) had sensitivities below 50% but specificities over 90%. Anosmia had a pooled sensitivity of 28.0% (95% CI 17.7% to 41.3%) and a specificity of 93.4% (95% CI 88.3% to 96.4%). Ageusia had a pooled sensitivity of 24.8% (95% CI 12.4% to 43.5%) and a specificity of 91.4% (95% CI 81.3% to 96.3%). Anosmia or ageusia had a pooled sensitivity of 41.0% (95% CI 27.0% to 56.6%) and a specificity of 90.5% (95% CI 81.2% to 95.4%). The pooled positive likelihood ratios of anosmia alone and anosmia or ageusia were 4.25 (95% CI 3.17 to 5.71) and 4.31 (95% CI 3.00 to 6.18) respectively, which is just below our arbitrary definition of a 'red flag', that is, a positive likelihood ratio of at least 5. The pooled positive likelihood ratio of ageusia alone was only 2.88 (95% CI 2.02 to 4.09). Only two studies assessed combinations of different signs and symptoms, mostly combining fever and cough with other symptoms. These combinations had a specificity above 80%, but at the cost of very low sensitivity (< 30%). AUTHORS' CONCLUSIONS The majority of individual signs and symptoms included in this review appear to have very poor diagnostic accuracy, although this should be interpreted in the context of selection bias and heterogeneity between studies. Based on currently available data, neither absence nor presence of signs or symptoms are accurate enough to rule in or rule out COVID-19. The presence of anosmia or ageusia may be useful as a red flag for COVID-19. The presence of fever or cough, given their high sensitivities, may also be useful to identify people for further testing. Prospective studies in an unselected population presenting to primary care or hospital outpatient settings, examining combinations of signs and symptoms to evaluate the syndromic presentation of COVID-19, are still urgently needed. Results from such studies could inform subsequent management decisions.
Collapse
Affiliation(s)
- Thomas Struyf
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham , UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Mariska Mg Leeflang
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
- Biomarker and Test Evaluation Programme (BiTE) , Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - René Spijker
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht , Netherlands
| | | | - Julie Domen
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | | | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| |
Collapse
|
26
|
Islam N, Salameh JP, Leeflang MM, Hooft L, McGrath TA, van der Pol CB, Frank RA, Kazi S, Prager R, Hare SS, Dennie C, Spijker R, Deeks JJ, Dinnes J, Jenniskens K, Korevaar DA, Cohen JF, Van den Bruel A, Takwoingi Y, van de Wijgert J, Wang J, McInnes MD. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2020; 11:CD013639. [PMID: 33242342 DOI: 10.1002/14651858.cd013639.pub3] [Citation(s) in RCA: 35] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
BACKGROUND The respiratory illness caused by SARS-CoV-2 infection continues to present diagnostic challenges. Early research showed thoracic (chest) imaging to be sensitive but not specific in the diagnosis of coronavirus disease 2019 (COVID-19). However, this is a rapidly developing field and these findings need to be re-evaluated in the light of new research. This is the first update of this 'living systematic review'. This update focuses on people suspected of having COVID-19 and excludes studies with only confirmed COVID-19 participants. OBJECTIVES To evaluate the diagnostic accuracy of thoracic imaging (computed tomography (CT), X-ray and ultrasound) in people with suspected COVID-19. SEARCH METHODS We searched the COVID-19 Living Evidence Database from the University of Bern, the Cochrane COVID-19 Study Register, The Stephen B. Thacker CDC Library, and repositories of COVID-19 publications through to 22 June 2020. We did not apply any language restrictions. SELECTION CRITERIA We included studies of all designs that recruited participants of any age group suspected to have COVID-19, and which reported estimates of test accuracy, or provided data from which estimates could be computed. When studies used a variety of reference standards, we retained the classification of participants as COVID-19 positive or negative as used in the study. DATA COLLECTION AND ANALYSIS We screened studies, extracted data, and assessed the risk of bias and applicability concerns using the QUADAS-2 domain-list independently, in duplicate. We categorised included studies into three groups based on classification of index test results: studies that reported specific criteria for index test positivity (group 1); studies that did not report specific criteria, but had the test reader(s) explicitly classify the imaging test result as either COVID-19 positive or negative (group 2); and studies that reported an overview of index test findings, without explicitly classifying the imaging test as either COVID-19 positive or negative (group 3). We presented the results of estimated sensitivity and specificity using paired forest plots, and summarised in tables. We used a bivariate meta-analysis model where appropriate. We presented uncertainty of the accuracy estimates using 95% confidence intervals (CIs). MAIN RESULTS We included 34 studies: 30 were cross-sectional studies with 8491 participants suspected of COVID-19, of which 4575 (54%) had a final diagnosis of COVID-19; four were case-control studies with 848 cases and controls in total, of which 464 (55%) had a final diagnosis of COVID-19. Chest CT was evaluated in 31 studies (8014 participants, 4224 (53%) cases), chest X-ray in three studies (1243 participants, 784 (63%) cases), and ultrasound of the lungs in one study (100 participants, 31 (31%) cases). Twenty-six per cent (9/34) of all studies were available only as preprints. Nineteen studies were conducted in Asia, 10 in Europe, four in North America and one in Australia. Sixteen studies included only adults, 15 studies included both adults and children and one included only children. Two studies did not report the ages of participants. Twenty-four studies included inpatients, four studies included outpatients, while the remaining six studies were conducted in unclear settings. The majority of included studies had a high or unclear risk of bias with respect to participant selection, index test, reference standard, and participant flow. For chest CT in suspected COVID-19 participants (31 studies, 8014 participants, 4224 (53%) cases) the sensitivity ranged from 57.4% to 100%, and specificity ranged from 0% to 96.0%. The pooled sensitivity of chest CT in suspected COVID-19 participants was 89.9% (95% CI 85.7 to 92.9) and the pooled specificity was 61.1% (95% CI 42.3 to 77.1). Sensitivity analyses showed that when the studies from China were excluded, the studies from other countries demonstrated higher specificity compared to the overall included studies. When studies that did not classify index tests as positive or negative for COVID-19 (group 3) were excluded, the remaining studies (groups 1 and 2) demonstrated higher specificity compared to the overall included studies. Sensitivity analyses limited to cross-sectional studies, or studies where at least two reverse transcriptase polymerase chain reaction (RT-PCR) tests were conducted if the first was negative, did not substantively alter the accuracy estimates. We did not identify publication status as a source of heterogeneity. For chest X-ray in suspected COVID-19 participants (3 studies, 1243 participants, 784 (63%) cases) the sensitivity ranged from 56.9% to 89.0% and specificity from 11.1% to 88.9%. The sensitivity and specificity of ultrasound of the lungs in suspected COVID-19 participants (1 study, 100 participants, 31 (31%) cases) were 96.8% and 62.3%, respectively. We could not perform a meta-analysis for chest X-ray or ultrasound due to the limited number of included studies. AUTHORS' CONCLUSIONS Our findings indicate that chest CT is sensitive and moderately specific for the diagnosis of COVID-19 in suspected patients, meaning that CT may have limited capability in differentiating SARS-CoV-2 infection from other causes of respiratory illness. However, we are limited in our confidence in these results due to the poor study quality and the heterogeneity of included studies. Because of limited data, accuracy estimates of chest X-ray and ultrasound of the lungs for the diagnosis of suspected COVID-19 cases should be carefully interpreted. Future diagnostic accuracy studies should pre-define positive imaging findings, include direct comparisons of the various modalities of interest on the same participant population, and implement improved reporting practices. Planned updates of this review will aim to: increase precision around the accuracy estimates for chest CT (ideally with low risk of bias studies); obtain further data to inform accuracy of chest X-rays and ultrasound; and obtain data to further fulfil secondary objectives (e.g. 'threshold' effects, comparing accuracy estimates across different imaging modalities) to inform the utility of imaging along different diagnostic pathways.
Collapse
Affiliation(s)
- Nayaar Islam
- Department of Radiology, University of Ottawa, Ottawa, Canada
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| | | | - Mariska Mg Leeflang
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | | | | | - Robert A Frank
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Sakib Kazi
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Ross Prager
- Department of Medicine, University of Ottawa, Ottawa, Canada
| | - Samanjit S Hare
- Department of Radiology, Royal Free London NHS Trust, London, UK
| | - Carole Dennie
- Department of Radiology, University of Ottawa, Ottawa, Canada
- Department of Medical Imaging, The Ottawa Hospital, Ottawa, Canada
| | - René Spijker
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
| | - Jonathan J Deeks
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Kevin Jenniskens
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Daniël A Korevaar
- Department of Respiratory Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - Jérémie F Cohen
- Obstetrical, Perinatal and Pediatric Epidemiology Research Team (EPOPé), Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité (CRESS), Inserm UMR1153, Université de Paris, Paris, France
| | | | - Yemisi Takwoingi
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Janneke van de Wijgert
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Institute of Infection, Veterinary, and Ecological Sciences, University of Liverpool, Liverpool, UK
| | - Junfeng Wang
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, Netherlands
| | - Matthew Df McInnes
- Department of Radiology, University of Ottawa, Ottawa, Canada
- Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
| |
Collapse
|
27
|
Stegeman I, Ochodo EA, Guleid F, Holtman GA, Yang B, Davenport C, Deeks JJ, Dinnes J, Dittrich S, Emperador D, Hooft L, Spijker R, Takwoingi Y, Van den Bruel A, Wang J, Langendam M, Verbakel JY, Leeflang MM. Routine laboratory testing to determine if a patient has COVID-19. Cochrane Database Syst Rev 2020; 11:CD013787. [PMID: 33211319 PMCID: PMC8078159 DOI: 10.1002/14651858.cd013787] [Citation(s) in RCA: 30] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/11/2023]
Abstract
BACKGROUND Specific diagnostic tests to detect severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and resulting COVID-19 disease are not always available and take time to obtain results. Routine laboratory markers such as white blood cell count, measures of anticoagulation, C-reactive protein (CRP) and procalcitonin, are used to assess the clinical status of a patient. These laboratory tests may be useful for the triage of people with potential COVID-19 to prioritize them for different levels of treatment, especially in situations where time and resources are limited. OBJECTIVES To assess the diagnostic accuracy of routine laboratory testing as a triage test to determine if a person has COVID-19. SEARCH METHODS On 4 May 2020 we undertook electronic searches in the Cochrane COVID-19 Study Register and the COVID-19 Living Evidence Database from the University of Bern, which is updated daily with published articles from PubMed and Embase and with preprints from medRxiv and bioRxiv. In addition, we checked repositories of COVID-19 publications. We did not apply any language restrictions. SELECTION CRITERIA We included both case-control designs and consecutive series of patients that assessed the diagnostic accuracy of routine laboratory testing as a triage test to determine if a person has COVID-19. The reference standard could be reverse transcriptase polymerase chain reaction (RT-PCR) alone; RT-PCR plus clinical expertise or and imaging; repeated RT-PCR several days apart or from different samples; WHO and other case definitions; and any other reference standard used by the study authors. DATA COLLECTION AND ANALYSIS Two review authors independently extracted data from each included study. They also assessed the methodological quality of the studies, using QUADAS-2. We used the 'NLMIXED' procedure in SAS 9.4 for the hierarchical summary receiver operating characteristic (HSROC) meta-analyses of tests for which we included four or more studies. To facilitate interpretation of results, for each meta-analysis we estimated summary sensitivity at the points on the SROC curve that corresponded to the median and interquartile range boundaries of specificities in the included studies. MAIN RESULTS We included 21 studies in this review, including 14,126 COVID-19 patients and 56,585 non-COVID-19 patients in total. Studies evaluated a total of 67 different laboratory tests. Although we were interested in the diagnotic accuracy of routine tests for COVID-19, the included studies used detection of SARS-CoV-2 infection through RT-PCR as reference standard. There was considerable heterogeneity between tests, threshold values and the settings in which they were applied. For some tests a positive result was defined as a decrease compared to normal vaues, for other tests a positive result was defined as an increase, and for some tests both increase and decrease may have indicated test positivity. None of the studies had either low risk of bias on all domains or low concerns for applicability for all domains. Only three of the tests evaluated had a summary sensitivity and specificity over 50%. These were: increase in interleukin-6, increase in C-reactive protein and lymphocyte count decrease. Blood count Eleven studies evaluated a decrease in white blood cell count, with a median specificity of 93% and a summary sensitivity of 25% (95% CI 8.0% to 27%; very low-certainty evidence). The 15 studies that evaluated an increase in white blood cell count had a lower median specificity and a lower corresponding sensitivity. Four studies evaluated a decrease in neutrophil count. Their median specificity was 93%, corresponding to a summary sensitivity of 10% (95% CI 1.0% to 56%; low-certainty evidence). The 11 studies that evaluated an increase in neutrophil count had a lower median specificity and a lower corresponding sensitivity. The summary sensitivity of an increase in neutrophil percentage (4 studies) was 59% (95% CI 1.0% to 100%) at median specificity (38%; very low-certainty evidence). The summary sensitivity of an increase in monocyte count (4 studies) was 13% (95% CI 6.0% to 26%) at median specificity (73%; very low-certainty evidence). The summary sensitivity of a decrease in lymphocyte count (13 studies) was 64% (95% CI 28% to 89%) at median specificity (53%; low-certainty evidence). Four studies that evaluated a decrease in lymphocyte percentage showed a lower median specificity and lower corresponding sensitivity. The summary sensitivity of a decrease in platelets (4 studies) was 19% (95% CI 10% to 32%) at median specificity (88%; low-certainty evidence). Liver function tests The summary sensitivity of an increase in alanine aminotransferase (9 studies) was 12% (95% CI 3% to 34%) at median specificity (92%; low-certainty evidence). The summary sensitivity of an increase in aspartate aminotransferase (7 studies) was 29% (95% CI 17% to 45%) at median specificity (81%) (low-certainty evidence). The summary sensitivity of a decrease in albumin (4 studies) was 21% (95% CI 3% to 67%) at median specificity (66%; low-certainty evidence). The summary sensitivity of an increase in total bilirubin (4 studies) was 12% (95% CI 3.0% to 34%) at median specificity (92%; very low-certainty evidence). Markers of inflammation The summary sensitivity of an increase in CRP (14 studies) was 66% (95% CI 55% to 75%) at median specificity (44%; very low-certainty evidence). The summary sensitivity of an increase in procalcitonin (6 studies) was 3% (95% CI 1% to 19%) at median specificity (86%; very low-certainty evidence). The summary sensitivity of an increase in IL-6 (four studies) was 73% (95% CI 36% to 93%) at median specificity (58%) (very low-certainty evidence). Other biomarkers The summary sensitivity of an increase in creatine kinase (5 studies) was 11% (95% CI 6% to 19%) at median specificity (94%) (low-certainty evidence). The summary sensitivity of an increase in serum creatinine (four studies) was 7% (95% CI 1% to 37%) at median specificity (91%; low-certainty evidence). The summary sensitivity of an increase in lactate dehydrogenase (4 studies) was 25% (95% CI 15% to 38%) at median specificity (72%; very low-certainty evidence). AUTHORS' CONCLUSIONS Although these tests give an indication about the general health status of patients and some tests may be specific indicators for inflammatory processes, none of the tests we investigated are useful for accurately ruling in or ruling out COVID-19 on their own. Studies were done in specific hospitalized populations, and future studies should consider non-hospital settings to evaluate how these tests would perform in people with milder symptoms.
Collapse
Affiliation(s)
- Inge Stegeman
- Department of Otorhinolaryngology & Head and Neck Surgery, University Medical Center Utrecht, Utrecht, Netherlands
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
- Brain Center Rudolf Magnus, University Medical Center Utrecht, Utrecht, Netherlands
| | - Eleanor A Ochodo
- Centre for Evidence-based Health Care, Department of Global Health, Faculty of Medicine and Health Sciences, Stellenbosch University, Cape Town, South Africa
- Centre for Global Health Research, Kenya Medical Research Institute, Kisumu, Kenya
| | - Fatuma Guleid
- KEMRI-Wellcome Trust Research Programme, Nairobi, Kenya
| | - Gea A Holtman
- Department of General Practice, University of Groningen, University Medical Centre Groningen, Groningen, Netherlands
| | - Bada Yang
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | | | | | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - René Spijker
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Junfeng Wang
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, Netherlands
| | - Miranda Langendam
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | - Jan Y Verbakel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Mariska Mg Leeflang
- Epidemiology and Data Science, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| |
Collapse
|
28
|
Salameh JP, Leeflang MM, Hooft L, Islam N, McGrath TA, van der Pol CB, Frank RA, Prager R, Hare SS, Dennie C, Spijker R, Deeks JJ, Dinnes J, Jenniskens K, Korevaar DA, Cohen JF, Van den Bruel A, Takwoingi Y, van de Wijgert J, Damen JA, Wang J, McInnes MD. Thoracic imaging tests for the diagnosis of COVID-19. Cochrane Database Syst Rev 2020; 9:CD013639. [PMID: 32997361 DOI: 10.1002/14651858.cd013639.pub2] [Citation(s) in RCA: 35] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
BACKGROUND The diagnosis of infection by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) presents major challenges. Reverse transcriptase polymerase chain reaction (RT-PCR) testing is used to diagnose a current infection, but its utility as a reference standard is constrained by sampling errors, limited sensitivity (71% to 98%), and dependence on the timing of specimen collection. Chest imaging tests are being used in the diagnosis of COVID-19 disease, or when RT-PCR testing is unavailable. OBJECTIVES To determine the diagnostic accuracy of chest imaging (computed tomography (CT), X-ray and ultrasound) in people with suspected or confirmed COVID-19. SEARCH METHODS We searched the COVID-19 Living Evidence Database from the University of Bern, the Cochrane COVID-19 Study Register, and The Stephen B. Thacker CDC Library. In addition, we checked repositories of COVID-19 publications. We did not apply any language restrictions. We conducted searches for this review iteration up to 5 May 2020. SELECTION CRITERIA We included studies of all designs that produce estimates of test accuracy or provide data from which estimates can be computed. We included two types of cross-sectional designs: a) where all patients suspected of the target condition enter the study through the same route and b) where it is not clear up front who has and who does not have the target condition, or where the patients with the target condition are recruited in a different way or from a different population from the patients without the target condition. When studies used a variety of reference standards, we included all of them. DATA COLLECTION AND ANALYSIS We screened studies and extracted data independently, in duplicate. We also assessed the risk of bias and applicability concerns independently, in duplicate, using the QUADAS-2 checklist and presented the results of estimated sensitivity and specificity, using paired forest plots, and summarised in tables. We used a hierarchical meta-analysis model where appropriate. We presented uncertainty of the accuracy estimates using 95% confidence intervals (CIs). MAIN RESULTS We included 84 studies, falling into two categories: studies with participants with confirmed diagnoses of COVID-19 at the time of recruitment (71 studies with 6331 participants) and studies with participants suspected of COVID-19 (13 studies with 1948 participants, including three case-control studies with 549 cases and controls). Chest CT was evaluated in 78 studies (8105 participants), chest X-ray in nine studies (682 COVID-19 cases), and chest ultrasound in two studies (32 COVID-19 cases). All evaluations of chest X-ray and ultrasound were conducted in studies with confirmed diagnoses only. Twenty-five per cent (21/84) of all studies were available only as preprints, 15/71 studies in the confirmed cases group and 6/13 of the studies in the suspected group. Among 71 studies that included confirmed cases, 41 studies had included symptomatic cases only, 25 studies had included cases regardless of their symptoms, five studies had included asymptomatic cases only, three of which included a combination of confirmed and suspected cases. Seventy studies were conducted in Asia, 2 in Europe, 2 in North America and one in South America. Fifty-one studies included inpatients while the remaining 24 studies were conducted in mixed or unclear settings. Risk of bias was high in most studies, mainly due to concerns about selection of participants and applicability. Among the 13 studies that included suspected cases, nine studies were conducted in Asia, and one in Europe. Seven studies included inpatients while the remaining three studies were conducted in mixed or unclear settings. In studies that included confirmed cases the pooled sensitivity of chest CT was 93.1% (95%CI: 90.2 - 95.0 (65 studies, 5759 cases); and for X-ray 82.1% (95%CI: 62.5 to 92.7 (9 studies, 682 cases). Heterogeneity judged by visual assessment of the ROC plots was considerable. Two studies evaluated the diagnostic accuracy of point-of-care ultrasound and both reported zero false negatives (with 10 and 22 participants having undergone ultrasound, respectively). These studies only reported True Positive and False Negative data, therefore it was not possible to pool and derive estimates of specificity. In studies that included suspected cases, the pooled sensitivity of CT was 86.2% (95%CI: 71.9 to 93.8 (13 studies, 2346 participants) and specificity was 18.1% (95%CI: 3.71 to 55.8). Heterogeneity judged by visual assessment of the forest plots was high. Chest CT may give approximately the same proportion of positive results for patients with and without a SARS-CoV-2 infection: the chances of getting a positive CT result are 86% (95% CI: 72 to 94) in patient with a SARS-CoV-2 infection and 82% (95% CI: 44 to 96) in patients without. AUTHORS' CONCLUSIONS The uncertainty resulting from the poor study quality and the heterogeneity of included studies limit our ability to confidently draw conclusions based on our results. Our findings indicate that chest CT is sensitive but not specific for the diagnosis of COVID-19 in suspected patients, meaning that CT may not be capable of differentiating SARS-CoV-2 infection from other causes of respiratory illness. This low specificity could also be the result of the poor sensitivity of the reference standard (RT-PCR), as CT could potentially be more sensitive than RT-PCR in some cases. Because of limited data, accuracy estimates of chest X-ray and ultrasound of the lungs for the diagnosis of COVID-19 should be carefully interpreted. Future diagnostic accuracy studies should avoid cases-only studies and pre-define positive imaging findings. Planned updates of this review will aim to: increase precision around the accuracy estimates for CT (ideally with low risk of bias studies); obtain further data to inform accuracy of chest X rays and ultrasound; and continue to search for studies that fulfil secondary objectives to inform the utility of imaging along different diagnostic pathways.
Collapse
Affiliation(s)
- Jean-Paul Salameh
- Department of Radiology, University of Ottawa, Ottawa, Canada
- Faculty of Health Sciences, Queen's University, Kingston, Canada
| | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Nayaar Islam
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | | | | | - Robert A Frank
- Department of Radiology, University of Ottawa, Ottawa, Canada
| | - Ross Prager
- Department of Medicine, University of Ottawa, Ottawa, Canada
| | - Samanjit S Hare
- Department of Radiology, Royal Free London NHS Trust, London, UK
| | - Carole Dennie
- Department of Radiology, University of Ottawa, Ottawa, Canada
- Department of Medical Imaging, The Ottawa Hospital, Ottawa, Canada
| | - René Spijker
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Kevin Jenniskens
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Daniël A Korevaar
- Department of Respiratory Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - Jérémie F Cohen
- Obstetrical, Perinatal and Pediatric Epidemiology Research Team (EPOPé), Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité (CRESS), Inserm UMR1153, Paris Descartes University, Paris, France
| | - Ann Van den Bruel
- NIHR Diagnostic Evidence Cooperative, University of Oxford, Oxford, UK
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Janneke van de Wijgert
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Institute of Infection, Veterinary, and Ecological Sciences, University of Liverpool, Liverpool, UK
| | - Johanna Aag Damen
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Junfeng Wang
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrehct, Netherlands
| | | |
Collapse
|
29
|
Dinnes J, Deeks JJ, Adriano A, Berhane S, Davenport C, Dittrich S, Emperador D, Takwoingi Y, Cunningham J, Beese S, Dretzke J, Ferrante di Ruffano L, Harris IM, Price MJ, Taylor-Phillips S, Hooft L, Leeflang MM, Spijker R, Van den Bruel A. Rapid, point-of-care antigen and molecular-based tests for diagnosis of SARS-CoV-2 infection. Cochrane Database Syst Rev 2020; 8:CD013705. [PMID: 32845525 PMCID: PMC8078202 DOI: 10.1002/14651858.cd013705] [Citation(s) in RCA: 341] [Impact Index Per Article: 85.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/18/2022]
Abstract
BACKGROUND Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) and the resulting COVID-19 pandemic present important diagnostic challenges. Several diagnostic strategies are available to identify or rule out current infection, identify people in need of care escalation, or to test for past infection and immune response. Point-of-care antigen and molecular tests to detect current SARS-CoV-2 infection have the potential to allow earlier detection and isolation of confirmed cases compared to laboratory-based diagnostic methods, with the aim of reducing household and community transmission. OBJECTIVES To assess the diagnostic accuracy of point-of-care antigen and molecular-based tests to determine if a person presenting in the community or in primary or secondary care has current SARS-CoV-2 infection. SEARCH METHODS On 25 May 2020 we undertook electronic searches in the Cochrane COVID-19 Study Register and the COVID-19 Living Evidence Database from the University of Bern, which is updated daily with published articles from PubMed and Embase and with preprints from medRxiv and bioRxiv. In addition, we checked repositories of COVID-19 publications. We did not apply any language restrictions. SELECTION CRITERIA We included studies of people with suspected current SARS-CoV-2 infection, known to have, or not to have SARS-CoV-2 infection, or where tests were used to screen for infection. We included test accuracy studies of any design that evaluated antigen or molecular tests suitable for a point-of-care setting (minimal equipment, sample preparation, and biosafety requirements, with results available within two hours of sample collection). We included all reference standards to define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction (RT-PCR) tests and established clinical diagnostic criteria). DATA COLLECTION AND ANALYSIS Two review authors independently screened studies and resolved any disagreements by discussion with a third review author. One review author independently extracted study characteristics, which were checked by a second review author. Two review authors independently extracted 2x2 contingency table data and assessed risk of bias and applicability of the studies using the QUADAS-2 tool. We present sensitivity and specificity, with 95% confidence intervals (CIs), for each test using paired forest plots. We pooled data using the bivariate hierarchical model separately for antigen and molecular-based tests, with simplifications when few studies were available. We tabulated available data by test manufacturer. MAIN RESULTS We included 22 publications reporting on a total of 18 study cohorts with 3198 unique samples, of which 1775 had confirmed SARS-CoV-2 infection. Ten studies took place in North America, two in South America, four in Europe, one in China and one was conducted internationally. We identified data for eight commercial tests (four antigen and four molecular) and one in-house antigen test. Five of the studies included were only available as preprints. We did not find any studies at low risk of bias for all quality domains and had concerns about applicability of results across all studies. We judged patient selection to be at high risk of bias in 50% of the studies because of deliberate over-sampling of samples with confirmed COVID-19 infection and unclear in seven out of 18 studies because of poor reporting. Sixteen (89%) studies used only a single, negative RT-PCR to confirm the absence of COVID-19 infection, risking missing infection. There was a lack of information on blinding of index test (n = 11), and around participant exclusions from analyses (n = 10). We did not observe differences in methodological quality between antigen and molecular test evaluations. Antigen tests Sensitivity varied considerably across studies (from 0% to 94%): the average sensitivity was 56.2% (95% CI 29.5 to 79.8%) and average specificity was 99.5% (95% CI 98.1% to 99.9%; based on 8 evaluations in 5 studies on 943 samples). Data for individual antigen tests were limited with no more than two studies for any test. Rapid molecular assays Sensitivity showed less variation compared to antigen tests (from 68% to 100%), average sensitivity was 95.2% (95% CI 86.7% to 98.3%) and specificity 98.9% (95% CI 97.3% to 99.5%) based on 13 evaluations in 11 studies of on 2255 samples. Predicted values based on a hypothetical cohort of 1000 people with suspected COVID-19 infection (with a prevalence of 10%) result in 105 positive test results including 10 false positives (positive predictive value 90%), and 895 negative results including 5 false negatives (negative predictive value 99%). Individual tests We calculated pooled results of individual tests for ID NOW (Abbott Laboratories) (5 evaluations) and Xpert Xpress (Cepheid Inc) (6 evaluations). Summary sensitivity for the Xpert Xpress assay (99.4%, 95% CI 98.0% to 99.8%) was 22.6 (95% CI 18.8 to 26.3) percentage points higher than that of ID NOW (76.8%, (95% CI 72.9% to 80.3%), whilst the specificity of Xpert Xpress (96.8%, 95% CI 90.6% to 99.0%) was marginally lower than ID NOW (99.6%, 95% CI 98.4% to 99.9%; a difference of -2.8% (95% CI -6.4 to 0.8)) AUTHORS' CONCLUSIONS: This review identifies early-stage evaluations of point-of-care tests for detecting SARS-CoV-2 infection, largely based on remnant laboratory samples. The findings currently have limited applicability, as we are uncertain whether tests will perform in the same way in clinical practice, and according to symptoms of COVID-19, duration of symptoms, or in asymptomatic people. Rapid tests have the potential to be used to inform triage of RT-PCR use, allowing earlier detection of those testing positive, but the evidence currently is not strong enough to determine how useful they are in clinical practice. Prospective and comparative evaluations of rapid tests for COVID-19 infection in clinically relevant settings are urgently needed. Studies should recruit consecutive series of eligible participants, including both those presenting for testing due to symptoms and asymptomatic people who may have come into contact with confirmed cases. Studies should clearly describe symptomatic status and document time from symptom onset or time since exposure. Point-of-care tests must be conducted on samples according to manufacturer instructions for use and be conducted at the point of care. Any future research study report should conform to the Standards for Reporting of Diagnostic Accuracy (STARD) guideline.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Ada Adriano
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Sarah Berhane
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | | | | | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jane Cunningham
- Global Malaria Programme, World Health Organization, Geneva, Switzerland
| | - Sophie Beese
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Janine Dretzke
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Lavinia Ferrante di Ruffano
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Isobel M Harris
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Malcolm J Price
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Sian Taylor-Phillips
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
- Biomarker and Test Evaluation Programme (BiTE), Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - René Spijker
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| |
Collapse
|
30
|
Struyf T, Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Leeflang MM, Spijker R, Hooft L, Emperador D, Dittrich S, Domen J, Horn SRA, Van den Bruel A. Signs and symptoms to determine if a patient presenting in primary care or hospital outpatient settings has COVID-19 disease. Cochrane Database Syst Rev 2020; 7:CD013665. [PMID: 32633856 PMCID: PMC7386785 DOI: 10.1002/14651858.cd013665] [Citation(s) in RCA: 224] [Impact Index Per Article: 56.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
BACKGROUND Some people with SARS-CoV-2 infection remain asymptomatic, whilst in others the infection can cause mild to moderate COVID-19 disease and COVID-19 pneumonia, leading some patients to require intensive care support and, in some cases, to death, especially in older adults. Symptoms such as fever or cough, and signs such as oxygen saturation or lung auscultation findings, are the first and most readily available diagnostic information. Such information could be used to either rule out COVID-19 disease, or select patients for further diagnostic testing. OBJECTIVES To assess the diagnostic accuracy of signs and symptoms to determine if a person presenting in primary care or to hospital outpatient settings, such as the emergency department or dedicated COVID-19 clinics, has COVID-19 disease or COVID-19 pneumonia. SEARCH METHODS On 27 April 2020, we undertook electronic searches in the Cochrane COVID-19 Study Register and the University of Bern living search database, which is updated daily with published articles from PubMed and Embase and with preprints from medRxiv and bioRxiv. In addition, we checked repositories of COVID-19 publications. We did not apply any language restrictions. SELECTION CRITERIA Studies were eligible if they included patients with suspected COVID-19 disease, or if they recruited known cases with COVID-19 disease and controls without COVID-19. Studies were eligible when they recruited patients presenting to primary care or hospital outpatient settings. Studies including patients who contracted SARS-CoV-2 infection while admitted to hospital were not eligible. The minimum eligible sample size of studies was 10 participants. All signs and symptoms were eligible for this review, including individual signs and symptoms or combinations. We accepted a range of reference standards including reverse transcription polymerase chain reaction (RT-PCR), clinical expertise, imaging, serology tests and World Health Organization (WHO) or other definitions of COVID-19. DATA COLLECTION AND ANALYSIS Pairs of review authors independently selected all studies, at both title and abstract stage and full-text stage. They resolved any disagreements by discussion with a third review author. Two review authors independently extracted data and resolved disagreements by discussion with a third review author. Two review authors independently assessed risk of bias using the QUADAS-2 checklist. Analyses were descriptive, presenting sensitivity and specificity in paired forest plots, in ROC (receiver operating characteristic) space and in dumbbell plots. We did not attempt meta-analysis due to the small number of studies, heterogeneity across studies and the high risk of bias. MAIN RESULTS We identified 16 studies including 7706 participants in total. Prevalence of COVID-19 disease varied from 5% to 38% with a median of 17%. There were no studies from primary care settings, although we did find seven studies in outpatient clinics (2172 participants), and four studies in the emergency department (1401 participants). We found data on 27 signs and symptoms, which fall into four different categories: systemic, respiratory, gastrointestinal and cardiovascular. No studies assessed combinations of different signs and symptoms and results were highly variable across studies. Most had very low sensitivity and high specificity; only six symptoms had a sensitivity of at least 50% in at least one study: cough, sore throat, fever, myalgia or arthralgia, fatigue, and headache. Of these, fever, myalgia or arthralgia, fatigue, and headache could be considered red flags (defined as having a positive likelihood ratio of at least 5) for COVID-19 as their specificity was above 90%, meaning that they substantially increase the likelihood of COVID-19 disease when present. Seven studies carried a high risk of bias for selection of participants because inclusion in the studies depended on the applicable testing and referral protocols, which included many of the signs and symptoms under study in this review. Five studies only included participants with pneumonia on imaging, suggesting that this is a highly selected population. In an additional four studies, we were unable to assess the risk for selection bias. These factors make it very difficult to determine the diagnostic properties of these signs and symptoms from the included studies. We also had concerns about the applicability of these results, since most studies included participants who were already admitted to hospital or presenting to hospital settings. This makes these findings less applicable to people presenting to primary care, who may have less severe illness and a lower prevalence of COVID-19 disease. None of the studies included any data on children, and only one focused specifically on older adults. We hope that future updates of this review will be able to provide more information about the diagnostic properties of signs and symptoms in different settings and age groups. AUTHORS' CONCLUSIONS The individual signs and symptoms included in this review appear to have very poor diagnostic properties, although this should be interpreted in the context of selection bias and heterogeneity between studies. Based on currently available data, neither absence nor presence of signs or symptoms are accurate enough to rule in or rule out disease. Prospective studies in an unselected population presenting to primary care or hospital outpatient settings, examining combinations of signs and symptoms to evaluate the syndromic presentation of COVID-19 disease, are urgently needed. Results from such studies could inform subsequent management decisions such as self-isolation or selecting patients for further diagnostic testing. We also need data on potentially more specific symptoms such as loss of sense of smell. Studies in older adults are especially important.
Collapse
Affiliation(s)
- Thomas Struyf
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
- Biomarker and Test Evaluation Programme (BiTE), Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - René Spijker
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | | | | | | | | | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| |
Collapse
|
31
|
Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Spijker R, Taylor-Phillips S, Adriano A, Beese S, Dretzke J, Ferrante di Ruffano L, Harris IM, Price MJ, Dittrich S, Emperador D, Hooft L, Leeflang MM, Van den Bruel A. Antibody tests for identification of current and past infection with SARS-CoV-2. Cochrane Database Syst Rev 2020; 6:CD013652. [PMID: 32584464 PMCID: PMC7387103 DOI: 10.1002/14651858.cd013652] [Citation(s) in RCA: 432] [Impact Index Per Article: 108.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
BACKGROUND The severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) virus and resulting COVID-19 pandemic present important diagnostic challenges. Several diagnostic strategies are available to identify current infection, rule out infection, identify people in need of care escalation, or to test for past infection and immune response. Serology tests to detect the presence of antibodies to SARS-CoV-2 aim to identify previous SARS-CoV-2 infection, and may help to confirm the presence of current infection. OBJECTIVES To assess the diagnostic accuracy of antibody tests to determine if a person presenting in the community or in primary or secondary care has SARS-CoV-2 infection, or has previously had SARS-CoV-2 infection, and the accuracy of antibody tests for use in seroprevalence surveys. SEARCH METHODS We undertook electronic searches in the Cochrane COVID-19 Study Register and the COVID-19 Living Evidence Database from the University of Bern, which is updated daily with published articles from PubMed and Embase and with preprints from medRxiv and bioRxiv. In addition, we checked repositories of COVID-19 publications. We did not apply any language restrictions. We conducted searches for this review iteration up to 27 April 2020. SELECTION CRITERIA We included test accuracy studies of any design that evaluated antibody tests (including enzyme-linked immunosorbent assays, chemiluminescence immunoassays, and lateral flow assays) in people suspected of current or previous SARS-CoV-2 infection, or where tests were used to screen for infection. We also included studies of people either known to have, or not to have SARS-CoV-2 infection. We included all reference standards to define the presence or absence of SARS-CoV-2 (including reverse transcription polymerase chain reaction tests (RT-PCR) and clinical diagnostic criteria). DATA COLLECTION AND ANALYSIS We assessed possible bias and applicability of the studies using the QUADAS-2 tool. We extracted 2x2 contingency table data and present sensitivity and specificity for each antibody (or combination of antibodies) using paired forest plots. We pooled data using random-effects logistic regression where appropriate, stratifying by time since post-symptom onset. We tabulated available data by test manufacturer. We have presented uncertainty in estimates of sensitivity and specificity using 95% confidence intervals (CIs). MAIN RESULTS We included 57 publications reporting on a total of 54 study cohorts with 15,976 samples, of which 8526 were from cases of SARS-CoV-2 infection. Studies were conducted in Asia (n = 38), Europe (n = 15), and the USA and China (n = 1). We identified data from 25 commercial tests and numerous in-house assays, a small fraction of the 279 antibody assays listed by the Foundation for Innovative Diagnostics. More than half (n = 28) of the studies included were only available as preprints. We had concerns about risk of bias and applicability. Common issues were use of multi-group designs (n = 29), inclusion of only COVID-19 cases (n = 19), lack of blinding of the index test (n = 49) and reference standard (n = 29), differential verification (n = 22), and the lack of clarity about participant numbers, characteristics and study exclusions (n = 47). Most studies (n = 44) only included people hospitalised due to suspected or confirmed COVID-19 infection. There were no studies exclusively in asymptomatic participants. Two-thirds of the studies (n = 33) defined COVID-19 cases based on RT-PCR results alone, ignoring the potential for false-negative RT-PCR results. We observed evidence of selective publication of study findings through omission of the identity of tests (n = 5). We observed substantial heterogeneity in sensitivities of IgA, IgM and IgG antibodies, or combinations thereof, for results aggregated across different time periods post-symptom onset (range 0% to 100% for all target antibodies). We thus based the main results of the review on the 38 studies that stratified results by time since symptom onset. The numbers of individuals contributing data within each study each week are small and are usually not based on tracking the same groups of patients over time. Pooled results for IgG, IgM, IgA, total antibodies and IgG/IgM all showed low sensitivity during the first week since onset of symptoms (all less than 30.1%), rising in the second week and reaching their highest values in the third week. The combination of IgG/IgM had a sensitivity of 30.1% (95% CI 21.4 to 40.7) for 1 to 7 days, 72.2% (95% CI 63.5 to 79.5) for 8 to 14 days, 91.4% (95% CI 87.0 to 94.4) for 15 to 21 days. Estimates of accuracy beyond three weeks are based on smaller sample sizes and fewer studies. For 21 to 35 days, pooled sensitivities for IgG/IgM were 96.0% (95% CI 90.6 to 98.3). There are insufficient studies to estimate sensitivity of tests beyond 35 days post-symptom onset. Summary specificities (provided in 35 studies) exceeded 98% for all target antibodies with confidence intervals no more than 2 percentage points wide. False-positive results were more common where COVID-19 had been suspected and ruled out, but numbers were small and the difference was within the range expected by chance. Assuming a prevalence of 50%, a value considered possible in healthcare workers who have suffered respiratory symptoms, we would anticipate that 43 (28 to 65) would be missed and 7 (3 to 14) would be falsely positive in 1000 people undergoing IgG/IgM testing at days 15 to 21 post-symptom onset. At a prevalence of 20%, a likely value in surveys in high-risk settings, 17 (11 to 26) would be missed per 1000 people tested and 10 (5 to 22) would be falsely positive. At a lower prevalence of 5%, a likely value in national surveys, 4 (3 to 7) would be missed per 1000 tested, and 12 (6 to 27) would be falsely positive. Analyses showed small differences in sensitivity between assay type, but methodological concerns and sparse data prevent comparisons between test brands. AUTHORS' CONCLUSIONS The sensitivity of antibody tests is too low in the first week since symptom onset to have a primary role for the diagnosis of COVID-19, but they may still have a role complementing other testing in individuals presenting later, when RT-PCR tests are negative, or are not done. Antibody tests are likely to have a useful role for detecting previous SARS-CoV-2 infection if used 15 or more days after the onset of symptoms. However, the duration of antibody rises is currently unknown, and we found very little data beyond 35 days post-symptom onset. We are therefore uncertain about the utility of these tests for seroprevalence surveys for public health management purposes. Concerns about high risk of bias and applicability make it likely that the accuracy of tests when used in clinical care will be lower than reported in the included studies. Sensitivity has mainly been evaluated in hospitalised patients, so it is unclear whether the tests are able to detect lower antibody levels likely seen with milder and asymptomatic COVID-19 disease. The design, execution and reporting of studies of the accuracy of COVID-19 tests requires considerable improvement. Studies must report data on sensitivity disaggregated by time since onset of symptoms. COVID-19-positive cases who are RT-PCR-negative should be included as well as those confirmed RT-PCR, in accordance with the World Health Organization (WHO) and China National Health Commission of the People's Republic of China (CDC) case definitions. We were only able to obtain data from a small proportion of available tests, and action is needed to ensure that all results of test evaluations are available in the public domain to prevent selective reporting. This is a fast-moving field and we plan ongoing updates of this living systematic review.
Collapse
Affiliation(s)
- Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - René Spijker
- Medical Library, Amsterdam UMC, University of Amsterdam, Amsterdam Public Health, Amsterdam, Netherlands
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Sian Taylor-Phillips
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- Division of Health Sciences, Warwick Medical School, University of Warwick, Coventry, UK
| | - Ada Adriano
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Sophie Beese
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Janine Dretzke
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Lavinia Ferrante di Ruffano
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Isobel M Harris
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Malcolm J Price
- Test Evaluation Research Group, Institute of Applied Health Research, University of Birmingham, Birmingham, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | | | | | - Lotty Hooft
- Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Mariska Mg Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam, Netherlands
- Biomarker and Test Evaluation Programme (BiTE), Amsterdam UMC, University of Amsterdam, Amsterdam, Netherlands
| | - Ann Van den Bruel
- Department of Public Health and Primary Care, KU Leuven, Leuven, Belgium
| |
Collapse
|
32
|
Deeks J, Dinnes J, Williams H. Sensitivity and specificity of SkinVision are likely to have been overestimated. J Eur Acad Dermatol Venereol 2020; 34:e582-e583. [DOI: 10.1111/jdv.16382] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2020] [Accepted: 03/16/2020] [Indexed: 12/24/2022]
Affiliation(s)
- J.J. Deeks
- Test Evaluation Research Group Institute of Applied Health Research University of Birmingham Birmingham UK
- NIHR Birmingham Biomedical Research Centre University Hospitals Birmingham NHS Foundation Trust and University of Birmingham Birmingham UK
| | - J. Dinnes
- Test Evaluation Research Group Institute of Applied Health Research University of Birmingham Birmingham UK
- NIHR Birmingham Biomedical Research Centre University Hospitals Birmingham NHS Foundation Trust and University of Birmingham Birmingham UK
| | - H.C. Williams
- Centre of Evidence‐Based Dermatology Nottingham University Hospitals NHS Trust Nottingham UK
| |
Collapse
|
33
|
McInnes MDF, Leeflang MMG, Salameh JP, McGrath TA, van der Pol CB, Frank RA, Prager R, Hare SS, Dennie C, Spijker R, Deeks JJ, Dinnes J, Jenniskens K, Korevaar DA, Cohen JF, Van den Bruel A, Takwoingi Y, van de Wijgert J, Damen JAAG, Hooft L. Imaging tests for the diagnosis of COVID-19. Cochrane Database of Systematic Reviews 2020. [DOI: 10.1002/14651858.cd013639] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Affiliation(s)
| | - Mariska MG Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics; Amsterdam University Medical Centers, University of Amsterdam; Amsterdam Netherlands
| | | | | | | | - Robert A Frank
- Department of Radiology; University of Ottawa; Ottawa Canada
| | - Ross Prager
- Department of Medicine; University of Ottawa; Ottawa Canada
| | - Samanjit S Hare
- Department of Radiology; Royal Free London NHS Trust; London UK
| | - Carole Dennie
- Department of Radiology; University of Ottawa; Ottawa Canada
- Department of Medical Imaging; The Ottawa Hospital; Ottawa Canada
| | - René Spijker
- Medical Library; Amsterdam UMC, University of Amsterdam, Amsterdam Public Health; Amsterdam Netherlands
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
| | - Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Kevin Jenniskens
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
| | - Daniël A Korevaar
- Department of Respiratory Medicine; Amsterdam UMC, University of Amsterdam; Amsterdam Netherlands
| | - Jérémie F Cohen
- Obstetrical, Perinatal and Pediatric Epidemiology Research Team (EPOPé); Centre de Recherche Épidémiologie et Statistique Sorbonne Paris Cité (CRESS), Inserm UMR1153, Paris Descartes University; Paris France
| | - Ann Van den Bruel
- NIHR Diagnostic Evidence Cooperative; University of Oxford; Oxford UK
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Janneke van de Wijgert
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
- Institute of Infection, Veterinary, and Ecological Sciences; University of Liverpool; Liverpool UK
| | - Johanna AAG Damen
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
| | - Lotty Hooft
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
| |
Collapse
|
34
|
Deeks JJ, Dinnes J, Takwoingi Y, Davenport C, Leeflang MMG, Spijker R, Hooft L, Van den Bruel A, Emperador D, Dittrich S. Diagnosis of SARS-CoV-2 infection and COVID-19: accuracy of signs and symptoms; molecular, antigen, and antibody tests; and routine laboratory markers. Cochrane Database Syst Rev 2020. [PMID: 32845525 DOI: 10.1002/14651858.cd013596] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Affiliation(s)
- Jonathan J Deeks
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Jacqueline Dinnes
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Yemisi Takwoingi
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Clare Davenport
- Test Evaluation Research Group, Institute of Applied Health Research; University of Birmingham; Birmingham UK
- NIHR Birmingham Biomedical Research Centre; University Hospitals Birmingham NHS Foundation Trust and University of Birmingham; Birmingham UK
| | - Mariska MG Leeflang
- Department of Clinical Epidemiology, Biostatistics and Bioinformatics; Amsterdam University Medical Centers, University of Amsterdam; Amsterdam Netherlands
- Biomarker and Test Evaluation Programme (BiTE); Amsterdam UMC, University of Amsterdam; Amsterdam Netherlands
| | - René Spijker
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
- Biomarker and Test Evaluation Programme (BiTE); Amsterdam UMC, University of Amsterdam; Amsterdam Netherlands
| | - Lotty Hooft
- Cochrane Netherlands; Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University; Utrecht Netherlands
| | - Ann Van den Bruel
- Department of Public Health and Primary Care; KU Leuven; Leuven Belgium
| | | | | |
Collapse
|
35
|
Freeman K, Dinnes J, Chuchu N, Takwoingi Y, Bayliss SE, Matin RN, Jain A, Walter FM, Williams HC, Deeks JJ. Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies. BMJ 2020; 368:m127. [PMID: 32041693 PMCID: PMC7190019 DOI: 10.1136/bmj.m127] [Citation(s) in RCA: 108] [Impact Index Per Article: 27.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/15/2022]
Abstract
OBJECTIVE To examine the validity and findings of studies that examine the accuracy of algorithm based smartphone applications ("apps") to assess risk of skin cancer in suspicious skin lesions. DESIGN Systematic review of diagnostic accuracy studies. DATA SOURCES Cochrane Central Register of Controlled Trials, MEDLINE, Embase, CINAHL, CPCI, Zetoc, Science Citation Index, and online trial registers (from database inception to 10 April 2019). ELIGIBILITY CRITERIA FOR SELECTING STUDIES Studies of any design that evaluated algorithm based smartphone apps to assess images of skin lesions suspicious for skin cancer. Reference standards included histological diagnosis or follow-up, and expert recommendation for further investigation or intervention. Two authors independently extracted data and assessed validity using QUADAS-2 (Quality Assessment of Diagnostic Accuracy Studies 2 tool). Estimates of sensitivity and specificity were reported for each app. RESULTS Nine studies that evaluated six different identifiable smartphone apps were included. Six verified results by using histology or follow-up (n=725 lesions), and three verified results by using expert recommendations (n=407 lesions). Studies were small and of poor methodological quality, with selective recruitment, high rates of unevaluable images, and differential verification. Lesion selection and image acquisition were performed by clinicians rather than smartphone users. Two CE (Conformit Europenne) marked apps are available for download. SkinScan was evaluated in a single study (n=15, five melanomas) with 0% sensitivity and 100% specificity for the detection of melanoma. SkinVision was evaluated in two studies (n=252, 61 malignant or premalignant lesions) and achieved a sensitivity of 80% (95% confidence interval 63% to 92%) and a specificity of 78% (67% to 87%) for the detection of malignant or premalignant lesions. Accuracy of the SkinVision app verified against expert recommendations was poor (three studies). CONCLUSIONS Current algorithm based smartphone apps cannot be relied on to detect all cases of melanoma or other skin cancers. Test performance is likely to be poorer than reported here when used in clinically relevant populations and by the intended users of the apps. The current regulatory process for awarding the CE marking for algorithm based apps does not provide adequate protection to the public. SYSTEMATIC REVIEW REGISTRATION PROSPERO CRD42016033595.
Collapse
Affiliation(s)
- Karoline Freeman
- Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
- Warwick Medical School, University of Warwick, Coventry, UK
| | - Jacqueline Dinnes
- Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Naomi Chuchu
- Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
- London School of Hygiene and Tropical Medicine, London, UK
| | - Yemisi Takwoingi
- Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| | - Sue E Bayliss
- Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
| | - Rubeta N Matin
- Department of Dermatology, Churchill Hospital, Oxford, UK
| | - Abhilash Jain
- Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
- Department of Plastic and Reconstructive Surgery, Imperial College Healthcare NHS Trust, St Mary's Hospital, London, UK
| | - Fiona M Walter
- Department of Public Health and Primary Care, University of Cambridge, Cambridge, UK
| | - Hywel C Williams
- Centre of Evidence Based Dermatology, University of Nottingham, Nottingham, UK
| | - Jonathan J Deeks
- Institute of Applied Health Research, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK
- NIHR Birmingham Biomedical Research Centre, University Hospitals Birmingham NHS Foundation Trust and University of Birmingham, Birmingham, UK
| |
Collapse
|
36
|
Dinnes J, Ferrante di Ruffano L, Takwoingi Y, Cheung ST, Nathan P, Matin RN, Chuchu N, Chan SA, Durack A, Bayliss SE, Gulati A, Patel L, Davenport C, Godfrey K, Subesinghe M, Traill Z, Deeks JJ, Williams HC. Ultrasound, CT, MRI, or PET-CT for staging and re-staging of adults with cutaneous melanoma. Cochrane Database Syst Rev 2019; 7:CD012806. [PMID: 31260100 PMCID: PMC6601698 DOI: 10.1002/14651858.cd012806.pub2] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
BACKGROUND Melanoma is one of the most aggressive forms of skin cancer, with the potential to metastasise to other parts of the body via the lymphatic system and the bloodstream. Melanoma accounts for a small percentage of skin cancer cases but is responsible for the majority of skin cancer deaths. Various imaging tests can be used with the aim of detecting metastatic spread of disease following a primary diagnosis of melanoma (primary staging) or on clinical suspicion of disease recurrence (re-staging). Accurate staging is crucial to ensuring that patients are directed to the most appropriate and effective treatment at different points on the clinical pathway. Establishing the comparative accuracy of ultrasound, computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET)-CT imaging for detection of nodal or distant metastases, or both, is critical to understanding if, how, and where on the pathway these tests might be used. OBJECTIVES Primary objectivesWe estimated accuracy separately according to the point in the clinical pathway at which imaging tests were used. Our objectives were:• to determine the diagnostic accuracy of ultrasound or PET-CT for detection of nodal metastases before sentinel lymph node biopsy in adults with confirmed cutaneous invasive melanoma; and• to determine the diagnostic accuracy of ultrasound, CT, MRI, or PET-CT for whole body imaging in adults with cutaneous invasive melanoma:○ for detection of any metastasis in adults with a primary diagnosis of melanoma (i.e. primary staging at presentation); and○ for detection of any metastasis in adults undergoing staging of recurrence of melanoma (i.e. re-staging prompted by findings on routine follow-up).We undertook separate analyses according to whether accuracy data were reported per patient or per lesion.Secondary objectivesWe sought to determine the diagnostic accuracy of ultrasound, CT, MRI, or PET-CT for whole body imaging (detection of any metastasis) in mixed or not clearly described populations of adults with cutaneous invasive melanoma.For study participants undergoing primary staging or re-staging (for possible recurrence), and for mixed or unclear populations, our objectives were:• to determine the diagnostic accuracy of ultrasound, CT, MRI, or PET-CT for detection of nodal metastases;• to determine the diagnostic accuracy of ultrasound, CT, MRI, or PET-CT for detection of distant metastases; and• to determine the diagnostic accuracy of ultrasound, CT, MRI, or PET-CT for detection of distant metastases according to metastatic site. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists as well as published systematic review articles. SELECTION CRITERIA We included studies of any design that evaluated ultrasound (with or without the use of fine needle aspiration cytology (FNAC)), CT, MRI, or PET-CT for staging of cutaneous melanoma in adults, compared with a reference standard of histological confirmation or imaging with clinical follow-up of at least three months' duration. We excluded studies reporting multiple applications of the same test in more than 10% of study participants. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2)). We estimated accuracy using the bivariate hierarchical method to produce summary sensitivities and specificities with 95% confidence and prediction regions. We undertook analysis of studies allowing direct and indirect comparison between tests. We examined heterogeneity between studies by visually inspecting the forest plots of sensitivity and specificity and summary receiver operating characteristic (ROC) plots. Numbers of identified studies were insufficient to allow formal investigation of potential sources of heterogeneity. MAIN RESULTS We included a total of 39 publications reporting on 5204 study participants; 34 studies reporting data per patient included 4980 study participants with 1265 cases of metastatic disease, and seven studies reporting data per lesion included 417 study participants with 1846 potentially metastatic lesions, 1061 of which were confirmed metastases. The risk of bias was low or unclear for all domains apart from participant flow. Concerns regarding applicability of the evidence were high or unclear for almost all domains. Participant selection from mixed or not clearly defined populations and poorly described application and interpretation of index tests were particularly problematic.The accuracy of imaging for detection of regional nodal metastases before sentinel lymph node biopsy (SLNB) was evaluated in 18 studies. In 11 studies (2614 participants; 542 cases), the summary sensitivity of ultrasound alone was 35.4% (95% confidence interval (CI) 17.0% to 59.4%) and specificity was 93.9% (95% CI 86.1% to 97.5%). Combining pre-SLNB ultrasound with FNAC revealed summary sensitivity of 18.0% (95% CI 3.58% to 56.5%) and specificity of 99.8% (95% CI 99.1% to 99.9%) (1164 participants; 259 cases). Four studies demonstrated lower sensitivity (10.2%, 95% CI 4.31% to 22.3%) and specificity (96.5%,95% CI 87.1% to 99.1%) for PET-CT before SLNB (170 participants, 49 cases). When these data are translated to a hypothetical cohort of 1000 people eligible for SLNB, 237 of whom have nodal metastases (median prevalence), the combination of ultrasound with FNAC potentially allows 43 people with nodal metastases to be triaged directly to adjuvant therapy rather than having SLNB first, at a cost of two people with false positive results (who are incorrectly managed). Those with a false negative ultrasound will be identified on subsequent SLNB.Limited test accuracy data were available for whole body imaging via PET-CT for primary staging or re-staging for disease recurrence, and none evaluated MRI. Twenty-four studies evaluated whole body imaging. Six of these studies explored primary staging following a confirmed diagnosis of melanoma (492 participants), three evaluated re-staging of disease following some clinical indication of recurrence (589 participants), and 15 included mixed or not clearly described population groups comprising participants at a number of different points on the clinical pathway and at varying stages of disease (1265 participants). Results for whole body imaging could not be translated to a hypothetical cohort of people due to paucity of data.Most of the studies (6/9) of primary disease or re-staging of disease considered PET-CT, two in comparison to CT alone, and three studies examined the use of ultrasound. No eligible evaluations of MRI in these groups were identified. All studies used histological reference standards combined with follow-up, and two included FNAC for some participants. Observed accuracy for detection of any metastases for PET-CT was higher for re-staging of disease (summary sensitivity from two studies: 92.6%, 95% CI 85.3% to 96.4%; specificity: 89.7%, 95% CI 78.8% to 95.3%; 153 participants; 95 cases) compared to primary staging (sensitivities from individual studies ranged from 30% to 47% and specificities from 73% to 88%), and was more sensitive than CT alone in both population groups, but participant numbers were very small.No conclusions can be drawn regarding routine imaging of the brain via MRI or CT. AUTHORS' CONCLUSIONS Review authors found a disappointing lack of evidence on the accuracy of imaging in people with a diagnosis of melanoma at different points on the clinical pathway. Studies were small and often reported data according to the number of lesions rather than the number of study participants. Imaging with ultrasound combined with FNAC before SLNB may identify around one-fifth of those with nodal disease, but confidence intervals are wide and further work is needed to establish cost-effectiveness. Much of the evidence for whole body imaging for primary staging or re-staging of disease is focused on PET-CT, and comparative data with CT or MRI are lacking. Future studies should go beyond diagnostic accuracy and consider the effects of different imaging tests on disease management. The increasing availability of adjuvant therapies for people with melanoma at high risk of disease spread at presentation will have a considerable impact on imaging services, yet evidence for the relative diagnostic accuracy of available tests is limited.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | | | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Seau Tak Cheung
- Dudley Hospitals Foundation Trust, Corbett HospitalDepartment of DermatologyWicarage RoadStourbridgeUKDY8 4JB
| | - Paul Nathan
- Mount Vernon HospitalMount Vernon Cancer CentreRickmansworth RoadNorthwoodUKHA6 2RN
| | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Sue Ann Chan
- City HospitalBirmingham Skin CentreDudley RdBirminghamUKB18 7QH
| | - Alana Durack
- Addenbrooke’s Hospital, Cambridge University Hospitals NHS Foundation TrustDermatologyHills RoadCambridgeUKCB2 0QQ
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Abha Gulati
- Barts Health NHS TrustDepartment of DermatologyWhitechapelLondonUKE11BB
| | - Lopa Patel
- Royal Stoke HospitalPlastic SurgeryStoke‐on‐TrentStaffordshireUKST4 6QG
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | - Manil Subesinghe
- King's College LondonCancer Imaging, School of Biomedical Engineering & Imaging SciencesLondonUK
| | - Zoe Traill
- Oxford University Hospitals NHS TrustChurchill Hospital Radiology DepartmentOxfordUK
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
37
|
Ferrante di Ruffano L, Dinnes J, Deeks JJ, Chuchu N, Bayliss SE, Davenport C, Takwoingi Y, Godfrey K, O'Sullivan C, Matin RN, Tehrani H, Williams HC. Optical coherence tomography for diagnosing skin cancer in adults. Cochrane Database Syst Rev 2018; 12:CD013189. [PMID: 30521690 PMCID: PMC6516952 DOI: 10.1002/14651858.cd013189] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/16/2022]
Abstract
BACKGROUND Early accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and squamous cell carcinoma (SCC) are high-risk skin cancers, which have the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised, with potential to infiltrate and damage surrounding tissue. Anxiety around missing early cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Optical coherence tomography (OCT) is a microscopic imaging technique, which magnifies the surface of a skin lesion using near-infrared light. Used in conjunction with clinical or dermoscopic examination of suspected skin cancer, or both, OCT may offer additional diagnostic information compared to other technologies. OBJECTIVES To determine the diagnostic accuracy of OCT for the detection of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, basal cell carcinoma (BCC), or cutaneous squamous cell carcinoma (cSCC) in adults. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA We included studies of any design evaluating OCT in adults with lesions suspicious for invasive melanoma and atypical intraepidermal melanocytic variants, BCC or cSCC, compared with a reference standard of histological confirmation or clinical follow-up. DATA COLLECTION AND ANALYSIS Two review authors independently extracted data using a standardised data extraction and quality assessment form (based on QUADAS-2). Our unit of analysis was lesions. Where possible, we estimated summary sensitivities and specificities using the bivariate hierarchical model. MAIN RESULTS We included five studies with 529 cutaneous lesions (282 malignant lesions) providing nine datasets for OCT, two for visual inspection alone, and two for visual inspection plus dermoscopy. Studies were of moderate to unclear quality, using data-driven thresholds for test positivity and giving poor accounts of reference standard interpretation and blinding. Studies may not have been representative of populations eligible for OCT in practice, for example due to high disease prevalence in study populations, and may not have reflected how OCT is used in practice, for example by using previously acquired OCT images.It was not possible to make summary statements regarding accuracy of detection of melanoma or of cSCC because of the paucity of studies, small sample sizes, and for melanoma differences in the OCT technologies used (high-definition versus conventional resolution OCT), and differences in the degree of testing performed prior to OCT (i.e. visual inspection alone or visual inspection plus dermoscopy).Pooled data from two studies using conventional swept-source OCT alongside visual inspection and dermoscopy for the detection of BCC estimated the sensitivity of OCT as 95% (95% confidence interval (CI) 91% to 97%) and specificity of 77% (95% CI 69% to 83%).When applied to a hypothetical population of 1000 lesions at the mean observed BCC prevalence of 60%, OCT would miss 31 BCCs (91 fewer than would be missed by visual inspection alone and 53 fewer than would be missed by visual inspection plus dermoscopy), and OCT would lead to 93 false-positive results for BCC (a reduction in unnecessary excisions of 159 compared to using visual inspection alone and of 87 compared to visual inspection plus dermoscopy). AUTHORS' CONCLUSIONS Insufficient data are available on the use of OCT for the detection of melanoma or cSCC. Initial data suggest conventional OCT may have a role for the diagnosis of BCC in clinically challenging lesions, with our meta-analysis showing a higher sensitivity and higher specificity when compared to visual inspection plus dermoscopy. However, the small number of studies and varying methodological quality means implications to guide practice cannot currently be drawn.Appropriately designed prospective comparative studies are required, given the paucity of data comparing OCT with dermoscopy and other similar diagnostic aids such as reflectance confocal microscopy.
Collapse
Affiliation(s)
| | - Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | | | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | - Hamid Tehrani
- Whiston HospitalDepartment of Plastic and Reconstructive SurgeryWarrington RoadLiverpoolUKL35 5DR
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
38
|
Dinnes J, Deeks JJ, Saleh D, Chuchu N, Bayliss SE, Patel L, Davenport C, Takwoingi Y, Godfrey K, Matin RN, Patalay R, Williams HC. Reflectance confocal microscopy for diagnosing cutaneous melanoma in adults. Cochrane Database Syst Rev 2018; 12:CD013190. [PMID: 30521681 PMCID: PMC6492459 DOI: 10.1002/14651858.cd013190] [Citation(s) in RCA: 33] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/03/2023]
Abstract
BACKGROUND Melanoma has one of the fastest rising incidence rates of any cancer. It accounts for a small percentage of skin cancer cases but is responsible for the majority of skin cancer deaths. Early detection and treatment is key to improving survival; however, anxiety around missing early cases needs to be balanced against appropriate levels of referral and excision of benign lesions. Used in conjunction with clinical or dermoscopic suspicion of malignancy, or both, reflectance confocal microscopy (RCM) may reduce unnecessary excisions without missing melanoma cases. OBJECTIVES To determine the diagnostic accuracy of reflectance confocal microscopy for the detection of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults with any lesion suspicious for melanoma and lesions that are difficult to diagnose, and to compare its accuracy with that of dermoscopy. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; and seven other databases. We studied reference lists and published systematic review articles. SELECTION CRITERIA Studies of any design that evaluated RCM alone, or RCM in comparison to dermoscopy, in adults with lesions suspicious for melanoma or atypical intraepidermal melanocytic variants, compared with a reference standard of either histological confirmation or clinical follow-up. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated summary sensitivities and specificities per algorithm and threshold using the bivariate hierarchical model. To compare RCM with dermoscopy, we grouped studies by population (defined by difficulty of lesion diagnosis) and combined data using hierarchical summary receiver operating characteristic (SROC) methods. Analysis of studies allowing direct comparison between tests was undertaken. To facilitate interpretation of results, we computed values of specificity at the point on the SROC curve with 90% sensitivity as this value lies within the estimates for the majority of analyses. We investigated the impact of using a purposely developed RCM algorithm and in-person test interpretation. MAIN RESULTS The search identified 18 publications reporting on 19 study cohorts with 2838 lesions (including 658 with melanoma), which provided 67 datasets for RCM and seven for dermoscopy. Studies were generally at high or unclear risk of bias across almost all domains and of high or unclear concern regarding applicability of the evidence. Selective participant recruitment, lack of blinding of the reference test to the RCM result, and differential verification were particularly problematic. Studies may not be representative of populations eligible for RCM, and test interpretation was often undertaken remotely from the patient and blinded to clinical information.Meta-analysis found RCM to be more accurate than dermoscopy in studies of participants with any lesion suspicious for melanoma and in participants with lesions that were more difficult to diagnose (equivocal lesion populations). Assuming a fixed sensitivity of 90% for both tests, specificities were 82% for RCM and 42% for dermoscopy for any lesion suspicious for melanoma (9 RCM datasets; 1452 lesions and 370 melanomas). For a hypothetical population of 1000 lesions at the median observed melanoma prevalence of 30%, this equated to a reduction in unnecessary excisions with RCM of 280 compared to dermoscopy, with 30 melanomas missed by both tests. For studies in equivocal lesions, specificities of 86% would be observed for RCM and 49% for dermoscopy (7 RCM datasets; 1177 lesions and 180 melanomas). At the median observed melanoma prevalence of 20%, this reduced unnecessary excisions by 296 with RCM compared with dermoscopy, with 20 melanomas missed by both tests. Across all populations, algorithms and thresholds assessed, the sensitivity and specificity of the Pellacani RCM score at a threshold of three or greater were estimated at 92% (95% confidence interval (CI) 87 to 95) for RCM and 72% (95% CI 62 to 81) for dermoscopy. AUTHORS' CONCLUSIONS RCM may have a potential role in clinical practice, particularly for the assessment of lesions that are difficult to diagnose using visual inspection and dermoscopy alone, where the evidence suggests that RCM may be both more sensitive and specific in comparison to dermoscopy. Given the paucity of data to allow comparison with dermoscopy, the results presented require further confirmation in prospective studies comparing RCM with dermoscopy in a real-world setting in a representative population.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Daniel Saleh
- Newcastle Hospitals NHS Trust, Royal Victoria InfirmaryNewcastle HospitalsNewcastleUK
- The University of Queensland, PA‐Southside Clinical UnitSchool of Clinical MedicineBrisbaneQueenslandAustralia
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Lopa Patel
- Royal Stoke HospitalPlastic SurgeryStoke‐on‐TrentStaffordshireUKST4 6QG
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | - Rakesh Patalay
- Guy's and St Thomas' NHS Foundation TrustDepartment of DermatologyDSLU, Cancer CentreGreat Maze PondLondonUKSE1 9RT
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
39
|
Dinnes J, Deeks JJ, Grainge MJ, Chuchu N, Ferrante di Ruffano L, Matin RN, Thomson DR, Wong KY, Aldridge RB, Abbott R, Fawzy M, Bayliss SE, Takwoingi Y, Davenport C, Godfrey K, Walter FM, Williams HC. Visual inspection for diagnosing cutaneous melanoma in adults. Cochrane Database Syst Rev 2018; 12:CD013194. [PMID: 30521684 PMCID: PMC6492463 DOI: 10.1002/14651858.cd013194] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/20/2022]
Abstract
BACKGROUND Melanoma has one of the fastest rising incidence rates of any cancer. It accounts for a small percentage of skin cancer cases but is responsible for the majority of skin cancer deaths. History-taking and visual inspection of a suspicious lesion by a clinician is usually the first in a series of 'tests' to diagnose skin cancer. Establishing the accuracy of visual inspection alone is critical to understating the potential contribution of additional tests to assist in the diagnosis of melanoma. OBJECTIVES To determine the diagnostic accuracy of visual inspection for the detection of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults with limited prior testing and in those referred for further evaluation of a suspicious lesion. Studies were separated according to whether the diagnosis was recorded face-to-face (in-person) or based on remote (image-based) assessment. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: CENTRAL; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA Test accuracy studies of any design that evaluated visual inspection in adults with lesions suspicious for melanoma, compared with a reference standard of either histological confirmation or clinical follow-up. We excluded studies reporting data for 'clinical diagnosis' where dermoscopy may or may not have been used. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated summary sensitivities and specificities per algorithm and threshold using the bivariate hierarchical model. We investigated the impact of: in-person test interpretation; use of a purposely developed algorithm to assist diagnosis; and observer expertise. MAIN RESULTS We included 49 publications reporting on a total of 51 study cohorts with 34,351 lesions (including 2499 cases), providing 134 datasets for visual inspection. Across almost all study quality domains, the majority of study reports provided insufficient information to allow us to judge the risk of bias, while in three of four domains that we assessed we scored concerns regarding applicability of study findings as 'high'. Selective participant recruitment, lack of detail regarding the threshold for deciding on a positive test result, and lack of detail on observer expertise were particularly problematic.Attempts to analyse studies by degree of prior testing were hampered by a lack of relevant information and by the restricted inclusion of lesions selected for biopsy or excision. Accuracy was generally much higher for in-person diagnosis compared to image-based evaluations (relative diagnostic odds ratio of 8.54, 95% CI 2.89 to 25.3, P < 0.001). Meta-analysis of in-person evaluations that could be clearly placed on the clinical pathway showed a general trade-off between sensitivity and specificity, with the highest sensitivity (92.4%, 95% CI 26.2% to 99.8%) and lowest specificity (79.7%, 95% CI 73.7% to 84.7%) observed in participants with limited prior testing (n = 3 datasets). Summary sensitivities were lower for those referred for specialist assessment but with much higher specificities (e.g. sensitivity 76.7%, 95% CI 61.7% to 87.1%) and specificity 95.7%, 95% CI 89.7% to 98.3%) for lesions selected for excision, n = 8 datasets). These differences may be related to differences in the spectrum of included lesions, differences in the definition of a positive test result, or to variations in observer expertise. We did not find clear evidence that accuracy is improved by the use of any algorithm to assist diagnosis in all settings. Attempts to examine the effect of observer expertise in melanoma diagnosis were hindered due to poor reporting. AUTHORS' CONCLUSIONS Visual inspection is a fundamental component of the assessment of a suspicious skin lesion; however, the evidence suggests that melanomas will be missed if visual inspection is used on its own. The evidence to support its accuracy in the range of settings in which it is used is flawed and very poorly reported. Although published algorithms do not appear to improve accuracy, there is insufficient evidence to suggest that the 'no algorithm' approach should be preferred in all settings. Despite the volume of research evaluating visual inspection, further prospective evaluation of the potential added value of using established algorithms according to the prior testing or diagnostic difficulty of lesions may be warranted.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Matthew J Grainge
- School of MedicineDivision of Epidemiology and Public HealthUniversity of NottinghamNottinghamUKNG7 2UH
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | | | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | | | - Kai Yuen Wong
- Oxford University Hospitals NHS Foundation TrustDepartment of Plastic and Reconstructive SurgeryOxfordUK
| | - Roger Benjamin Aldridge
- NHS Lothian/University of EdinburghDepartment of Plastic Surgery25/6 India StreetEdinburghUKEH3 6HE
| | - Rachel Abbott
- University Hospital of WalesWelsh Institute of DermatologyHeath ParkCardiffUKCF14 4XW
| | - Monica Fawzy
- Norfolk and Norwich University Hospital NHS TrustDepartment of Plastic and Reconstructive SurgeryColney LaneNorwichUKNR4 7UY
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | - Fiona M Walter
- University of CambridgePublic Health & Primary CareStrangeways Research Laboratory, Worts CausewayCambridgeUKCB1 8RN
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
40
|
Ferrante di Ruffano L, Dinnes J, Chuchu N, Bayliss SE, Takwoingi Y, Davenport C, Matin RN, O'Sullivan C, Roskell D, Deeks JJ, Williams HC. Exfoliative cytology for diagnosing basal cell carcinoma and other skin cancers in adults. Cochrane Database Syst Rev 2018; 12:CD013187. [PMID: 30521689 PMCID: PMC6517175 DOI: 10.1002/14651858.cd013187] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
BACKGROUND Early accurate detection of all skin cancer types is essential to guide appropriate management, reduce morbidity and improve survival. Basal cell carcinoma (BCC) is usually localised to the skin but has potential to infiltrate and damage surrounding tissue, while cutaneous squamous cell carcinoma (cSCC) and melanoma have a much higher potential to metastasise and ultimately lead to death. Exfoliative cytology is a non-invasive test that uses the Tzanck smear technique to identify disease by examining the structure of cells obtained from scraped samples. This simple procedure is a less invasive diagnostic test than a skin biopsy, and for BCC it has the potential to provide an immediate diagnosis that avoids an additional clinic visit to receive skin biopsy results. This may benefit patients scheduled for either Mohs micrographic surgery or non-surgical treatments such as radiotherapy. A cytology scrape can never give the same information as a skin biopsy, however, so it is important to better understand in which skin cancer situations it may be helpful. OBJECTIVES To determine the diagnostic accuracy of exfoliative cytology for detecting basal cell carcinoma (BCC) in adults, and to compare its accuracy with that of standard diagnostic practice (visual inspection with or without dermoscopy). Secondary objectives were: to determine the diagnostic accuracy of exfoliative cytology for detecting cSCC, invasive melanoma and atypical intraepidermal melanocytic variants, and any other skin cancer; and for each of these secondary conditions to compare the accuracy of exfoliative cytology with visual inspection with or without dermoscopy in direct test comparisons; and to determine the effect of observer experience. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We also studied the reference lists of published systematic review articles. SELECTION CRITERIA Studies evaluating exfoliative cytology in adults with lesions suspicious for BCC, cSCC or melanoma, compared with a reference standard of histological confirmation. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). Where possible we estimated summary sensitivities and specificities using the bivariate hierarchical model. MAIN RESULTS We synthesised the results of nine studies contributing a total of 1655 lesions to our analysis, including 1120 BCCs (14 datasets), 41 cSCCs (amongst 401 lesions in 2 datasets), and 10 melanomas (amongst 200 lesions in 1 dataset). Three of these datasets (one each for BCC, melanoma and any malignant condition) were derived from one study that also performed a direct comparison with dermoscopy. Studies were of moderate to poor quality, providing inadequate descriptions of participant selection, thresholds used to make cytological and histological diagnoses, and blinding. Reporting of participants' prior referral pathways was particularly poor, as were descriptions of the cytodiagnostic criteria used to make diagnoses. No studies evaluated the use of exfoliative cytology as a primary diagnostic test for detecting BCC or other skin cancers in lesions suspicious for skin cancer. Pooled data from seven studies using standard cytomorphological criteria (but various stain methods) to detect BCC in participants with a high clinical suspicion of BCC estimated the sensitivity and specificity of exfoliative cytology as 97.5% (95% CI 94.5% to 98.9%) and 90.1% (95% CI 81.1% to 95.1%). respectively. When applied to a hypothetical population of 1000 clinically suspected BCC lesions with a median observed BCC prevalence of 86%, exfoliative cytology would miss 21 BCCs and would lead to 14 false positive diagnoses of BCC. No false positive cases were histologically confirmed to be melanoma. Insufficient data are available to make summary statements regarding the accuracy of exfoliative cytology to detect melanoma or cSCC, or its accuracy compared to dermoscopy. AUTHORS' CONCLUSIONS The utility of exfoliative cytology for the primary diagnosis of skin cancer is unknown, as all included studies focused on the use of this technique for confirming strongly suspected clinical diagnoses. For the confirmation of BCC in lesions with a high clinical suspicion, there is evidence of high sensitivity and specificity. Since decisions to treat low-risk BCCs are unlikely in practice to require diagnostic confirmation given that clinical suspicion is already high, exfoliative cytology might be most useful for cases of BCC where the treatments being contemplated require a tissue diagnosis (e.g. radiotherapy). The small number of included studies, poor reporting and varying methodological quality prevent us from drawing strong conclusions to guide clinical practice. Despite insufficient data on the use of cytology for cSCC or melanoma, it is unlikely that cytology would be useful in these scenarios since preservation of the architecture of the whole lesion that would be available from a biopsy provides crucial diagnostic information. Given the paucity of good quality data, appropriately designed prospective comparative studies may be required to evaluate both the diagnostic value of exfoliative cytology by comparison to dermoscopy, and its confirmatory value in adequately reported populations with a high probability of BCC scheduled for further treatment requiring a tissue diagnosis.
Collapse
Affiliation(s)
| | - Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | | | - Derek Roskell
- Oxford University Hospitals NHS TrustDepartment of Cellular PathologyJohn Radcliffe HospitalHeadingtonOxfordUKOX3 9DU
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
41
|
Ferrante di Ruffano L, Takwoingi Y, Dinnes J, Chuchu N, Bayliss SE, Davenport C, Matin RN, Godfrey K, O'Sullivan C, Gulati A, Chan SA, Durack A, O'Connell S, Gardiner MD, Bamber J, Deeks JJ, Williams HC. Computer-assisted diagnosis techniques (dermoscopy and spectroscopy-based) for diagnosing skin cancer in adults. Cochrane Database Syst Rev 2018; 12:CD013186. [PMID: 30521691 PMCID: PMC6517147 DOI: 10.1002/14651858.cd013186] [Citation(s) in RCA: 44] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
BACKGROUND Early accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and cutaneous squamous cell carcinoma (cSCC) are high-risk skin cancers which have the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised with potential to infiltrate and damage surrounding tissue. Anxiety around missing early curable cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Computer-assisted diagnosis (CAD) systems use artificial intelligence to analyse lesion data and arrive at a diagnosis of skin cancer. When used in unreferred settings ('primary care'), CAD may assist general practitioners (GPs) or other clinicians to more appropriately triage high-risk lesions to secondary care. Used alongside clinical and dermoscopic suspicion of malignancy, CAD may reduce unnecessary excisions without missing melanoma cases. OBJECTIVES To determine the accuracy of CAD systems for diagnosing cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, BCC or cSCC in adults, and to compare its accuracy with that of dermoscopy. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials (CENTRAL); MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA Studies of any design that evaluated CAD alone, or in comparison with dermoscopy, in adults with lesions suspicious for melanoma or BCC or cSCC, and compared with a reference standard of either histological confirmation or clinical follow-up. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated summary sensitivities and specificities separately by type of CAD system, using the bivariate hierarchical model. We compared CAD with dermoscopy using (a) all available CAD data (indirect comparisons), and (b) studies providing paired data for both tests (direct comparisons). We tested the contribution of human decision-making to the accuracy of CAD diagnoses in a sensitivity analysis by removing studies that gave CAD results to clinicians to guide diagnostic decision-making. MAIN RESULTS We included 42 studies, 24 evaluating digital dermoscopy-based CAD systems (Derm-CAD) in 23 study cohorts with 9602 lesions (1220 melanomas, at least 83 BCCs, 9 cSCCs), providing 32 datasets for Derm-CAD and seven for dermoscopy. Eighteen studies evaluated spectroscopy-based CAD (Spectro-CAD) in 16 study cohorts with 6336 lesions (934 melanomas, 163 BCC, 49 cSCCs), providing 32 datasets for Spectro-CAD and six for dermoscopy. These consisted of 15 studies using multispectral imaging (MSI), two studies using electrical impedance spectroscopy (EIS) and one study using diffuse-reflectance spectroscopy. Studies were incompletely reported and at unclear to high risk of bias across all domains. Included studies inadequately address the review question, due to an abundance of low-quality studies, poor reporting, and recruitment of highly selected groups of participants.Across all CAD systems, we found considerable variation in the hardware and software technologies used, the types of classification algorithm employed, methods used to train the algorithms, and which lesion morphological features were extracted and analysed across all CAD systems, and even between studies evaluating CAD systems. Meta-analysis found CAD systems had high sensitivity for correct identification of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in highly selected populations, but with low and very variable specificity, particularly for Spectro-CAD systems. Pooled data from 22 studies estimated the sensitivity of Derm-CAD for the detection of melanoma as 90.1% (95% confidence interval (CI) 84.0% to 94.0%) and specificity as 74.3% (95% CI 63.6% to 82.7%). Pooled data from eight studies estimated the sensitivity of multispectral imaging CAD (MSI-CAD) as 92.9% (95% CI 83.7% to 97.1%) and specificity as 43.6% (95% CI 24.8% to 64.5%). When applied to a hypothetical population of 1000 lesions at the mean observed melanoma prevalence of 20%, Derm-CAD would miss 20 melanomas and would lead to 206 false-positive results for melanoma. MSI-CAD would miss 14 melanomas and would lead to 451 false diagnoses for melanoma. Preliminary findings suggest CAD systems are at least as sensitive as assessment of dermoscopic images for the diagnosis of invasive melanoma and atypical intraepidermal melanocytic variants. We are unable to make summary statements about the use of CAD in unreferred populations, or its accuracy in detecting keratinocyte cancers, or its use in any setting as a diagnostic aid, because of the paucity of studies. AUTHORS' CONCLUSIONS In highly selected patient populations all CAD types demonstrate high sensitivity, and could prove useful as a back-up for specialist diagnosis to assist in minimising the risk of missing melanomas. However, the evidence base is currently too poor to understand whether CAD system outputs translate to different clinical decision-making in practice. Insufficient data are available on the use of CAD in community settings, or for the detection of keratinocyte cancers. The evidence base for individual systems is too limited to draw conclusions on which might be preferred for practice. Prospective comparative studies are required that evaluate the use of already evaluated CAD systems as diagnostic aids, by comparison to face-to-face dermoscopy, and in participant populations that are representative of those in which the test would be used in practice.
Collapse
Affiliation(s)
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
| | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | | | - Abha Gulati
- Barts Health NHS TrustDepartment of DermatologyWhitechapelLondonUKE11BB
| | - Sue Ann Chan
- City HospitalBirmingham Skin CentreDudley RdBirminghamUKB18 7QH
| | - Alana Durack
- Addenbrooke’s Hospital, Cambridge University Hospitals NHS Foundation TrustDermatologyHills RoadCambridgeUKCB2 0QQ
| | - Susan O'Connell
- Cardiff and Vale University Health BoardCEDAR Healthcare Technology Research CentreCardiff Medicentre, University Hospital of Wales, Heath Park CampusCardiffWalesUKCF144UJ
| | | | - Jeffrey Bamber
- Institute of Cancer Research and The Royal Marsden NHS Foundation TrustJoint Department of Physics15 Cotswold RoadSuttonUKSM2 5NG
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchEdgbaston CampusBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
42
|
Dinnes J, Deeks JJ, Chuchu N, Saleh D, Bayliss SE, Takwoingi Y, Davenport C, Patel L, Matin RN, O'Sullivan C, Patalay R, Williams HC. Reflectance confocal microscopy for diagnosing keratinocyte skin cancers in adults. Cochrane Database Syst Rev 2018; 12:CD013191. [PMID: 30521687 PMCID: PMC6516892 DOI: 10.1002/14651858.cd013191] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/09/2023]
Abstract
BACKGROUND Early accurate detection of all skin cancer types is important to guide appropriate management and improve morbidity and survival. Basal cell carcinoma (BCC) is usually a localised skin cancer but with potential to infiltrate and damage surrounding tissue, whereas cutaneous squamous cell carcinoma (cSCC) and melanoma are higher risk skin cancers with the potential to metastasise and ultimately lead to death. When used in conjunction with clinical or dermoscopic suspicion of malignancy, or both, reflectance confocal microscopy (RCM) may help to identify cancers eligible for non-surgical treatment without the need for a diagnostic biopsy, particularly in people with suspected BCC. Any potential benefit must be balanced against the risk of any misdiagnoses. OBJECTIVES To determine the diagnostic accuracy of RCM for the detection of BCC, cSCC, or any skin cancer in adults with any suspicious lesion and lesions that are difficult to diagnose (equivocal); and to compare its accuracy with that of usual practice (visual inspection or dermoscopy, or both). SEARCH METHODS We undertook a comprehensive search of the following databases from inception to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA Studies of any design that evaluated the accuracy of RCM alone, or RCM in comparison to visual inspection or dermoscopy, or both, in adults with lesions suspicious for skin cancer compared with a reference standard of either histological confirmation or clinical follow-up, or both. DATA COLLECTION AND ANALYSIS Two review authors independently extracted data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated summary sensitivities and specificities using the bivariate hierarchical model. For computation of likely numbers of true-positive, false-positive, false-negative, and true-negative findings in the 'Summary of findings' tables, we applied summary sensitivity and specificity estimates to lower quartile, median and upper quartiles of the prevalence observed in the study groups. We also investigated the impact of observer experience. MAIN RESULTS The review included 10 studies reporting on 11 study cohorts. All 11 cohorts reported data for the detection of BCC, including 2037 lesions (464 with BCC); and four cohorts reported data for the detection of cSCC, including 834 lesions (71 with cSCC). Only one study also reported data for the detection of BCC or cSCC using dermoscopy, limiting comparisons between RCM and dermoscopy. Studies were at high or unclear risk of bias across almost all methodological quality domains, and were of high or unclear concern regarding applicability of the evidence. Selective participant recruitment, unclear blinding of the reference test, and exclusions due to image quality or technical difficulties were observed. It was unclear whether studies were representative of populations eligible for testing with RCM, and test interpretation was often undertaken using images, remotely from the participant and the interpreter blinded to clinical information that would normally be available in practice.Meta-analysis found RCM to be more sensitive but less specific for the detection of BCC in studies of participants with equivocal lesions (sensitivity 94%, 95% confidence interval (CI) 79% to 98%; specificity 85%, 95% CI 72% to 92%; 3 studies) compared to studies that included any suspicious lesion (sensitivity 76%, 95% CI 45% to 92%; specificity 95%, 95% CI 66% to 99%; 4 studies), although CIs were wide. At the median prevalence of disease of 12.5% observed in studies including any suspicious lesion, applying these results to a hypothetical population of 1000 lesions results in 30 BCCs missed with 44 false-positive results (lesions misdiagnosed as BCCs). At the median prevalence of disease of 15% observed in studies of equivocal lesions, nine BCCs would be missed with 128 false-positive results in a population of 1000 lesions. Across both sets of studies, up to 15% of these false-positive lesions were observed to be melanomas mistaken for BCCs. There was some suggestion of higher sensitivities in studies with more experienced observers. Summary sensitivity and specificity could not be estimated for the detection of cSCC due to paucity of data. AUTHORS' CONCLUSIONS There is insufficient evidence for the use of RCM for the diagnosis of BCC or cSCC in either population group. A possible role for RCM in clinical practice is as a tool to avoid diagnostic biopsies in lesions with a relatively high clinical suspicion of BCC. The potential for, and consequences of, misclassification of other skin cancers such as melanoma as BCCs requires further research. Importantly, data are lacking that compare RCM to standard clinical practice (with or without dermoscopy).
Collapse
Affiliation(s)
- Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Daniel Saleh
- Newcastle Hospitals NHS Trust, Royal Victoria InfirmaryNewcastle HospitalsNewcastleUK
- The University of Queensland, PA‐Southside Clinical UnitSchool of Clinical MedicineBrisbaneQueenslandAustralia
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Lopa Patel
- Royal Stoke HospitalPlastic SurgeryStoke‐on‐TrentStaffordshireUKST4 6QG
| | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | | | - Rakesh Patalay
- Guy's and St Thomas' NHS Foundation TrustDepartment of DermatologyDSLU, Cancer CentreGreat Maze PondLondonUKSE1 9RT
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
43
|
Dinnes J, Bamber J, Chuchu N, Bayliss SE, Takwoingi Y, Davenport C, Godfrey K, O'Sullivan C, Matin RN, Deeks JJ, Williams HC. High-frequency ultrasound for diagnosing skin cancer in adults. Cochrane Database Syst Rev 2018; 12:CD013188. [PMID: 30521683 PMCID: PMC6516989 DOI: 10.1002/14651858.cd013188] [Citation(s) in RCA: 35] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/14/2022]
Abstract
BACKGROUND Early, accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and squamous cell carcinoma (SCC) are high-risk skin cancers with the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised, with potential to infiltrate and damage surrounding tissue. Anxiety around missing early curable cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Ultrasound is a non-invasive imaging technique that relies on the measurement of sound wave reflections from the tissues of the body. At lower frequencies, the deeper structures of the body such as the internal organs can be visualised, while high-frequency ultrasound (HFUS) with transducer frequencies of 20 MHz or more has a much lower depth of tissue penetration but produces a higher resolution image of tissues and structures closer to the skin surface. Used in conjunction with clinical and/or dermoscopic examination of suspected skin cancer, HFUS may offer additional diagnostic information compared to other technologies. OBJECTIVES To assess the diagnostic accuracy of HFUS to assist in the diagnosis of a) cutaneous invasive melanoma and atypical intraepidermal melanocytic variants, b) cutaneous squamous cell carcinoma (cSCC), and c) basal cell carcinoma (BCC) in adults. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists as well as published systematic review articles. SELECTION CRITERIA Studies evaluating HFUS (20 MHz or more) in adults with lesions suspicious for melanoma, cSCC or BCC versus a reference standard of histological confirmation or clinical follow-up. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). Due to scarcity of data and the poor quality of studies, we did not undertake a meta-analysis for this review. For illustrative purposes, we plot estimates of sensitivity and specificity on coupled forest plots. MAIN RESULTS We included six studies, providing 29 datasets: 20 for diagnosis of melanoma (1125 lesions and 242 melanomas) and 9 for diagnosis of BCC (993 lesions and 119 BCCs). We did not identify any data relating to the diagnosis of cSCC.Studies were generally poorly reported, limiting judgements of methodological quality. Half the studies did not set out to establish test accuracy, and all should be considered preliminary evaluations of the potential usefulness of HFUS. There were particularly high concerns for applicability of findings due to selective study populations and data-driven thresholds for test positivity. Studies reporting qualitative assessments of HFUS images excluded up to 22% of lesions (including some melanomas) due to lack of visualisation in the test.Derived sensitivities for qualitative HFUS characteristics were at least 83% (95% CI 75% to 90%) for the detection of melanoma; the combination of three features (lesions appearing hypoechoic, homogenous and well defined) demonstrating 100% sensitivity in two studies (lower limits of the 95% CIs were 94% and 82%), with variable corresponding specificities of 33% (95% CI 20% to 48%) and 73% (95% CI 57% to 85%), respectively. Quantitative measurement of HFUS outputs in two studies enabled decision thresholds to be set to achieve 100% sensitivity; specificities were 93% (95% CI 77% to 99%) and 65% (95% CI 51% to 76%). It was not possible to make summary statements regarding HFUS accuracy for the diagnosis of BCC due to highly variable sensitivities and specificities. AUTHORS' CONCLUSIONS Insufficient data are available on the potential value of HFUS in the diagnosis of melanoma or BCC. Given the between-study heterogeneity, unclear to low methodological quality and limited volume of evidence, we cannot draw any implications for practice. The main value of the preliminary studies included may be in providing guidance on the possible components of new diagnostic rules for diagnosis of melanoma or BCC using HFUS that will require future evaluation. A prospective evaluation of HFUS added to visual inspection and dermoscopy alone in a standard healthcare setting, with a clearly defined and representative population of participants, would be required for a full and proper evaluation of accuracy.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jeffrey Bamber
- Institute of Cancer Research and The Royal Marsden NHS Foundation TrustJoint Department of Physics15 Cotswold RoadSuttonUKSM2 5NG
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | | | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
44
|
Dinnes J, Deeks JJ, Chuchu N, Matin RN, Wong KY, Aldridge RB, Durack A, Gulati A, Chan SA, Johnston L, Bayliss SE, Leonardi‐Bee J, Takwoingi Y, Davenport C, O'Sullivan C, Tehrani H, Williams HC. Visual inspection and dermoscopy, alone or in combination, for diagnosing keratinocyte skin cancers in adults. Cochrane Database Syst Rev 2018; 12:CD011901. [PMID: 30521688 PMCID: PMC6516870 DOI: 10.1002/14651858.cd011901.pub2] [Citation(s) in RCA: 23] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Abstract
BACKGROUND Early accurate detection of all skin cancer types is important to guide appropriate management, to reduce morbidity and to improve survival. Basal cell carcinoma (BCC) is almost always a localised skin cancer with potential to infiltrate and damage surrounding tissue, whereas a minority of cutaneous squamous cell carcinomas (cSCCs) and invasive melanomas are higher-risk skin cancers with the potential to metastasise and cause death. Dermoscopy has become an important tool to assist specialist clinicians in the diagnosis of melanoma, and is increasingly used in primary-care settings. Dermoscopy is a precision-built handheld illuminated magnifier that allows more detailed examination of the skin down to the level of the superficial dermis. Establishing the value of dermoscopy over and above visual inspection for the diagnosis of BCC or cSCC in primary- and secondary-care settings is critical to understanding its potential contribution to appropriate skin cancer triage, including referral of higher-risk cancers to secondary care, the identification of low-risk skin cancers that might be treated in primary care and to provide reassurance to those with benign skin lesions who can be safely discharged. OBJECTIVES To determine the diagnostic accuracy of visual inspection and dermoscopy, alone or in combination, for the detection of (a) BCC and (b) cSCC, in adults. We separated studies according to whether the diagnosis was recorded face-to-face (in person) or based on remote (image-based) assessment. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA Studies of any design that evaluated visual inspection or dermoscopy or both in adults with lesions suspicious for skin cancer, compared with a reference standard of either histological confirmation or clinical follow-up. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic thresholds were missing. We estimated accuracy using hierarchical summary ROC methods. We undertook analysis of studies allowing direct comparison between tests. To facilitate interpretation of results, we computed values of sensitivity at the point on the SROC curve with 80% fixed specificity and values of specificity with 80% fixed sensitivity. We investigated the impact of in-person test interpretation; use of a purposely-developed algorithm to assist diagnosis; and observer expertise. MAIN RESULTS We included 24 publications reporting on 24 study cohorts, providing 27 visual inspection datasets (8805 lesions; 2579 malignancies) and 33 dermoscopy datasets (6855 lesions; 1444 malignancies). The risk of bias was mainly low for the index test (for dermoscopy evaluations) and reference standard domains, particularly for in-person evaluations, and high or unclear for participant selection, application of the index test for visual inspection and for participant flow and timing. We scored concerns about the applicability of study findings as of 'high' or 'unclear' concern for almost all studies across all domains assessed. Selective participant recruitment, lack of reproducibility of diagnostic thresholds and lack of detail on observer expertise were particularly problematic.The detection of BCC was reported in 28 datasets; 15 on an in-person basis and 13 image-based. Analysis of studies by prior testing of participants and according to observer expertise was not possible due to lack of data. Studies were primarily conducted in participants referred for specialist assessment of lesions with available histological classification. We found no clear differences in accuracy between dermoscopy studies undertaken in person and those which evaluated images. The lack of effect observed may be due to other sources of heterogeneity, including variations in the types of skin lesion studied, in dermatoscopes used, or in the use of algorithms and varying thresholds for deciding on a positive test result.Meta-analysis found in-person evaluations of dermoscopy (7 evaluations; 4683 lesions and 363 BCCs) to be more accurate than visual inspection alone for the detection of BCC (8 evaluations; 7017 lesions and 1586 BCCs), with a relative diagnostic odds ratio (RDOR) of 8.2 (95% confidence interval (CI) 3.5 to 19.3; P < 0.001). This corresponds to predicted differences in sensitivity of 14% (93% versus 79%) at a fixed specificity of 80% and predicted differences in specificity of 22% (99% versus 77%) at a fixed sensitivity of 80%. We observed very similar results for the image-based evaluations.When applied to a hypothetical population of 1000 lesions, of which 170 are BCC (based on median BCC prevalence across studies), an increased sensitivity of 14% from dermoscopy would lead to 24 fewer BCCs missed, assuming 166 false positive results from both tests. A 22% increase in specificity from dermoscopy with sensitivity fixed at 80% would result in 183 fewer unnecessary excisions, assuming 34 BCCs missed for both tests. There was not enough evidence to assess the use of algorithms or structured checklists for either visual inspection or dermoscopy.Insufficient data were available to draw conclusions on the accuracy of either test for the detection of cSCCs. AUTHORS' CONCLUSIONS Dermoscopy may be a valuable tool for the diagnosis of BCC as an adjunct to visual inspection of a suspicious skin lesion following a thorough history-taking including assessment of risk factors for keratinocyte cancer. The evidence primarily comes from secondary-care (referred) populations and populations with pigmented lesions or mixed lesion types. There is no clear evidence supporting the use of currently-available formal algorithms to assist dermoscopy diagnosis.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | - Kai Yuen Wong
- Oxford University Hospitals NHS Foundation TrustDepartment of Plastic and Reconstructive SurgeryOxfordUK
| | - Roger Benjamin Aldridge
- NHS Lothian/University of EdinburghDepartment of Plastic Surgery25/6 India StreetEdinburghUKEH3 6HE
| | - Alana Durack
- Addenbrooke’s Hospital, Cambridge University Hospitals NHS Foundation TrustDermatologyHills RoadCambridgeUKCB2 0QQ
| | - Abha Gulati
- Barts Health NHS TrustDepartment of DermatologyWhitechapelLondonUKE11BB
| | - Sue Ann Chan
- City HospitalBirmingham Skin CentreDudley RdBirminghamUKB18 7QH
| | - Louise Johnston
- NIHR Diagnostic Evidence Co‐operative Newcastle2nd Floor William Leech Building (Rm M2.061) Institute of Cellular Medicine Newcastle UniversityFramlington PlaceNewcastle upon TyneUKNE2 4HH
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Jo Leonardi‐Bee
- The University of NottinghamDivision of Epidemiology and Public HealthClinical Sciences BuildingNottingham City Hospital NHS Trust Campus, Hucknall RoadNottinghamUKNG5 1PB
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | | | - Hamid Tehrani
- Whiston HospitalDepartment of Plastic and Reconstructive SurgeryWarrington RoadLiverpoolUKL35 5DR
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
45
|
Chuchu N, Dinnes J, Takwoingi Y, Matin RN, Bayliss SE, Davenport C, Moreau JF, Bassett O, Godfrey K, O'Sullivan C, Walter FM, Motley R, Deeks JJ, Williams HC. Teledermatology for diagnosing skin cancer in adults. Cochrane Database Syst Rev 2018; 12:CD013193. [PMID: 30521686 PMCID: PMC6517019 DOI: 10.1002/14651858.cd013193] [Citation(s) in RCA: 46] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
Abstract
BACKGROUND Early accurate detection of all skin cancer types is essential to guide appropriate management and to improve morbidity and survival. Melanoma and squamous cell carcinoma (SCC) are high-risk skin cancers which have the potential to metastasise and ultimately lead to death, whereas basal cell carcinoma (BCC) is usually localised with potential to infiltrate and damage surrounding tissue. Anxiety around missing early curable cases needs to be balanced against inappropriate referral and unnecessary excision of benign lesions. Teledermatology provides a way for generalist clinicians to access the opinion of a specialist dermatologist for skin lesions that they consider to be suspicious without referring the patients through the normal referral pathway. Teledermatology consultations can be 'store-and-forward' with electronic digital images of a lesion sent to a dermatologist for review at a later time, or can be live and interactive consultations using videoconferencing to connect the patient, referrer and dermatologist in real time. OBJECTIVES To determine the diagnostic accuracy of teledermatology for the detection of any skin cancer (melanoma, BCC or cutaneous squamous cell carcinoma (cSCC)) in adults, and to compare its accuracy with that of in-person diagnosis. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: Cochrane Central Register of Controlled Trials, MEDLINE, Embase, CINAHL, CPCI, Zetoc, Science Citation Index, US National Institutes of Health Ongoing Trials Register, NIHR Clinical Research Network Portfolio Database and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA Studies evaluating skin cancer diagnosis for teledermatology alone, or in comparison with face-to-face diagnosis by a specialist clinician, compared with a reference standard of histological confirmation or clinical follow-up and expert opinion. We also included studies evaluating the referral accuracy of teledermatology compared with a reference standard of face-to-face diagnosis by a specialist clinician. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where there were information related to the target condition of any skin cancer missing. Data permitting, we estimated summary sensitivities and specificities using the bivariate hierarchical model. Due to the scarcity of data, we undertook no covariate investigations for this review. For illustrative purposes, we plotted estimates of sensitivity and specificity on coupled forest plots for diagnostic threshold and target condition under consideration. MAIN RESULTS The review included 22 studies reporting diagnostic accuracy data for 4057 lesions and 879 malignant cases (16 studies) and referral accuracy data for reported data for 1449 lesions and 270 'positive' cases as determined by the reference standard face-to-face decision (six studies). Methodological quality was variable with poor reporting hindering assessment. The overall risk of bias was high or unclear for participant selection, reference standard, and participant flow and timing in at least half of all studies; the majority were at low risk of bias for the index test. The applicability of study findings were of high or unclear concern for most studies in all domains assessed due to the recruitment of participants from secondary care settings or specialist clinics rather than from primary or community-based settings in which teledermatology is more likely to be used and due to the acquisition of lesion images by dermatologists or in specialist imaging units rather than by primary care clinicians.Seven studies provided data for the primary target condition of any skin cancer (1588 lesions and 638 malignancies). For the correct diagnosis of lesions as malignant using photographic images, summary sensitivity was 94.9% (95% confidence interval (CI) 90.1% to 97.4%) and summary specificity was 84.3% (95% CI 48.5% to 96.8%) (from four studies). Individual study estimates using dermoscopic images or a combination of photographic and dermoscopic images generally suggested similarly high sensitivities with highly variable specificities. Limited comparative data suggested similar diagnostic accuracy between teledermatology assessment and in-person diagnosis by a dermatologist; however, data were too scarce to draw firm conclusions. For the detection of invasive melanoma or atypical intraepidermal melanocytic variants both sensitivities and specificities were more variable. Sensitivities ranged from 59% (95% CI 42% to 74%) to 100% (95% CI 48% to 100%) and specificities from 30% (95% CI 22% to 40%) to 100% (95% CI 93% to 100%), with reported diagnostic thresholds including the correct diagnosis of melanoma, classification of lesions as 'atypical' or 'typical, and the decision to refer or to excise a lesion.Referral accuracy data comparing teledermatology against a face-to-face reference standard suggested good agreement for lesions considered to require some positive action by face-to-face assessment (sensitivities of over 90%). For lesions considered of less concern when assessed face-to-face (e.g. for lesions not recommended for excision or referral), agreement was more variable with teledermatology specificities ranging from 57% (95% CI 39% to 73%) to 100% (95% CI 86% to 100%), suggesting that remote assessment is more likely recommend excision, referral or follow-up compared to in-person decisions. AUTHORS' CONCLUSIONS Studies were generally small and heterogeneous and methodological quality was difficult to judge due to poor reporting. Bearing in mind concerns regarding the applicability of study participants and of lesion image acquisition in specialist settings, our results suggest that teledermatology can correctly identify the majority of malignant lesions. Using a more widely defined threshold to identify 'possibly' malignant cases or lesions that should be considered for excision is likely to appropriately triage those lesions requiring face-to-face assessment by a specialist. Despite the increasing use of teledermatology on an international level, the evidence base to support its ability to accurately diagnose lesions and to triage lesions from primary to secondary care is lacking and further prospective and pragmatic evaluation is needed.
Collapse
Affiliation(s)
- Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Jacqueline F Moreau
- University of Pittsburgh Medical CenterInternal MedicineDepartment of Medicine, Office of EducationUPMC Montefiore Hospital, N715PittsburghUSAPA, 15213
| | - Oliver Bassett
- Addenbrooke's HospitalPlastic SurgeryHills RoadCambridgeUKCB2 0QQ
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | | | - Fiona M Walter
- University of CambridgePublic Health & Primary CareStrangeways Research Laboratory, Worts CausewayCambridgeUKCB1 8RN
| | - Richard Motley
- University Hospital of WalesWelsh Institute of DermatologyHeath ParkCardiffUKCF14 4XW
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
46
|
Dinnes J, Deeks JJ, Chuchu N, Ferrante di Ruffano L, Matin RN, Thomson DR, Wong KY, Aldridge RB, Abbott R, Fawzy M, Bayliss SE, Grainge MJ, Takwoingi Y, Davenport C, Godfrey K, Walter FM, Williams HC. Dermoscopy, with and without visual inspection, for diagnosing melanoma in adults. Cochrane Database Syst Rev 2018; 12:CD011902. [PMID: 30521682 PMCID: PMC6517096 DOI: 10.1002/14651858.cd011902.pub2] [Citation(s) in RCA: 62] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
BACKGROUND Melanoma has one of the fastest rising incidence rates of any cancer. It accounts for a small percentage of skin cancer cases but is responsible for the majority of skin cancer deaths. Although history-taking and visual inspection of a suspicious lesion by a clinician are usually the first in a series of 'tests' to diagnose skin cancer, dermoscopy has become an important tool to assist diagnosis by specialist clinicians and is increasingly used in primary care settings. Dermoscopy is a magnification technique using visible light that allows more detailed examination of the skin compared to examination by the naked eye alone. Establishing the additive value of dermoscopy over and above visual inspection alone across a range of observers and settings is critical to understanding its contribution for the diagnosis of melanoma and to future understanding of the potential role of the growing number of other high-resolution image analysis techniques. OBJECTIVES To determine the diagnostic accuracy of dermoscopy alone, or when added to visual inspection of a skin lesion, for the detection of cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults. We separated studies according to whether the diagnosis was recorded face-to-face (in-person), or based on remote (image-based), assessment. SEARCH METHODS We undertook a comprehensive search of the following databases from inception up to August 2016: CENTRAL; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA Studies of any design that evaluated dermoscopy in adults with lesions suspicious for melanoma, compared with a reference standard of either histological confirmation or clinical follow-up. Data on the accuracy of visual inspection, to allow comparisons of tests, was included only if reported in the included studies of dermoscopy. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). We contacted authors of included studies where information related to the target condition or diagnostic threshold were missing. We estimated accuracy using hierarchical summary receiver operating characteristic (SROC),methods. Analysis of studies allowing direct comparison between tests was undertaken. To facilitate interpretation of results, we computed values of sensitivity at the point on the SROC curve with 80% fixed specificity and values of specificity with 80% fixed sensitivity. We investigated the impact of in-person test interpretation; use of a purposely developed algorithm to assist diagnosis; observer expertise; and dermoscopy training. MAIN RESULTS We included a total of 104 study publications reporting on 103 study cohorts with 42,788 lesions (including 5700 cases), providing 354 datasets for dermoscopy. The risk of bias was mainly low for the index test and reference standard domains and mainly high or unclear for participant selection and participant flow. Concerns regarding the applicability of study findings were largely scored as 'high' concern in three of four domains assessed. Selective participant recruitment, lack of reproducibility of diagnostic thresholds and lack of detail on observer expertise were particularly problematic.The accuracy of dermoscopy for the detection of invasive melanoma or atypical intraepidermal melanocytic variants was reported in 86 datasets; 26 for evaluations conducted in person (dermoscopy added to visual inspection), and 60 for image-based evaluations (diagnosis based on interpretation of dermoscopic images). Analyses of studies by prior testing revealed no obvious effect on accuracy; analyses were hampered by the lack of studies in primary care, lack of relevant information and the restricted inclusion of lesions selected for biopsy or excision. Accuracy was higher for in-person diagnosis compared to image-based evaluations (relative diagnostic odds ratio (RDOR) 4.6, 95% confidence interval (CI) 2.4 to 9.0; P < 0.001).We compared accuracy for (a), in-person evaluations of dermoscopy (26 evaluations; 23,169 lesions and 1664 melanomas),versus visual inspection alone (13 evaluations; 6740 lesions and 459 melanomas), and for (b), image-based evaluations of dermoscopy (60 evaluations; 13,475 lesions and 2851 melanomas),versus image-based visual inspection (11 evaluations; 1740 lesions and 305 melanomas). For both comparisons, meta-analysis found dermoscopy to be more accurate than visual inspection alone, with RDORs of (a), 4.7 (95% CI 3.0 to 7.5; P < 0.001), and (b), 5.6 (95% CI 3.7 to 8.5; P < 0.001). For a), the predicted difference in sensitivity at a fixed specificity of 80% was 16% (95% CI 8% to 23%; 92% for dermoscopy + visual inspection versus 76% for visual inspection), and predicted difference in specificity at a fixed sensitivity of 80% was 20% (95% CI 7% to 33%; 95% for dermoscopy + visual inspection versus 75% for visual inspection). For b) the predicted differences in sensitivity was 34% (95% CI 24% to 46%; 81% for dermoscopy versus 47% for visual inspection), at a fixed specificity of 80%, and predicted difference in specificity was 40% (95% CI 27% to 57%; 82% for dermoscopy versus 42% for visual inspection), at a fixed sensitivity of 80%.Using the median prevalence of disease in each set of studies ((a), 12% for in-person and (b), 24% for image-based), for a hypothetical population of 1000 lesions, an increase in sensitivity of (a), 16% (in-person), and (b), 34% (image-based), from using dermoscopy at a fixed specificity of 80% equates to a reduction in the number of melanomas missed of (a), 19 and (b), 81 with (a), 176 and (b), 152 false positive results. An increase in specificity of (a), 20% (in-person), and (b), 40% (image-based), at a fixed sensitivity of 80% equates to a reduction in the number of unnecessary excisions from using dermoscopy of (a), 176 and (b), 304 with (a), 24 and (b), 48 melanomas missed.The use of a named or published algorithm to assist dermoscopy interpretation (as opposed to no reported algorithm or reported use of pattern analysis), had no significant impact on accuracy either for in-person (RDOR 1.4, 95% CI 0.34 to 5.6; P = 0.17), or image-based (RDOR 1.4, 95% CI 0.60 to 3.3; P = 0.22), evaluations. This result was supported by subgroup analysis according to algorithm used. We observed higher accuracy for observers reported as having high experience and for those classed as 'expert consultants' in comparison to those considered to have less experience in dermoscopy, particularly for image-based evaluations. Evidence for the effect of dermoscopy training on test accuracy was very limited but suggested associated improvements in sensitivity. AUTHORS' CONCLUSIONS Despite the observed limitations in the evidence base, dermoscopy is a valuable tool to support the visual inspection of a suspicious skin lesion for the detection of melanoma and atypical intraepidermal melanocytic variants, particularly in referred populations and in the hands of experienced users. Data to support its use in primary care are limited, however, it may assist in triaging suspicious lesions for urgent referral when employed by suitably trained clinicians. Formal algorithms may be of most use for dermoscopy training purposes and for less expert observers, however reliable data comparing approaches using dermoscopy in person are lacking.
Collapse
Affiliation(s)
- Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | | | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | | | - Kai Yuen Wong
- Oxford University Hospitals NHS Foundation TrustDepartment of Plastic and Reconstructive SurgeryOxfordUK
| | - Roger Benjamin Aldridge
- NHS Lothian/University of EdinburghDepartment of Plastic Surgery25/6 India StreetEdinburghUKEH3 6HE
| | - Rachel Abbott
- University Hospital of WalesWelsh Institute of DermatologyHeath ParkCardiffUKCF14 4XW
| | - Monica Fawzy
- Norfolk and Norwich University Hospital NHS TrustDepartment of Plastic and Reconstructive SurgeryColney LaneNorwichUKNR4 7UY
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Matthew J Grainge
- School of MedicineDivision of Epidemiology and Public HealthUniversity of NottinghamNottinghamUKNG7 2UH
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | - Fiona M Walter
- University of CambridgePublic Health & Primary CareStrangeways Research Laboratory, Worts CausewayCambridgeUKCB1 8RN
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
47
|
Chuchu N, Takwoingi Y, Dinnes J, Matin RN, Bassett O, Moreau JF, Bayliss SE, Davenport C, Godfrey K, O'Connell S, Jain A, Walter FM, Deeks JJ, Williams HC. Smartphone applications for triaging adults with skin lesions that are suspicious for melanoma. Cochrane Database Syst Rev 2018; 12:CD013192. [PMID: 30521685 PMCID: PMC6517294 DOI: 10.1002/14651858.cd013192] [Citation(s) in RCA: 44] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
BACKGROUND Melanoma accounts for a small proportion of all skin cancer cases but is responsible for most skin cancer-related deaths. Early detection and treatment can improve survival. Smartphone applications are readily accessible and potentially offer an instant risk assessment of the likelihood of malignancy so that the right people seek further medical attention from a clinician for more detailed assessment of the lesion. There is, however, a risk that melanomas will be missed and treatment delayed if the application reassures the user that their lesion is low risk. OBJECTIVES To assess the diagnostic accuracy of smartphone applications to rule out cutaneous invasive melanoma and atypical intraepidermal melanocytic variants in adults with concerns about suspicious skin lesions. SEARCH METHODS We undertook a comprehensive search of the following databases from inception to August 2016: Cochrane Central Register of Controlled Trials; MEDLINE; Embase; CINAHL; CPCI; Zetoc; Science Citation Index; US National Institutes of Health Ongoing Trials Register; NIHR Clinical Research Network Portfolio Database; and the World Health Organization International Clinical Trials Registry Platform. We studied reference lists and published systematic review articles. SELECTION CRITERIA Studies of any design evaluating smartphone applications intended for use by individuals in a community setting who have lesions that might be suspicious for melanoma or atypical intraepidermal melanocytic variants versus a reference standard of histological confirmation or clinical follow-up and expert opinion. DATA COLLECTION AND ANALYSIS Two review authors independently extracted all data using a standardised data extraction and quality assessment form (based on QUADAS-2). Due to scarcity of data and poor quality of studies, we did not perform a meta-analysis for this review. For illustrative purposes, we plotted estimates of sensitivity and specificity on coupled forest plots for each application under consideration. MAIN RESULTS This review reports on two cohorts of lesions published in two studies. Both studies were at high risk of bias from selective participant recruitment and high rates of non-evaluable images. Concerns about applicability of findings were high due to inclusion only of lesions already selected for excision in a dermatology clinic setting, and image acquisition by clinicians rather than by smartphone app users.We report data for five mobile phone applications and 332 suspicious skin lesions with 86 melanomas across the two studies. Across the four artificial intelligence-based applications that classified lesion images (photographs) as melanomas (one application) or as high risk or 'problematic' lesions (three applications) using a pre-programmed algorithm, sensitivities ranged from 7% (95% CI 2% to 16%) to 73% (95% CI 52% to 88%) and specificities from 37% (95% CI 29% to 46%) to 94% (95% CI 87% to 97%). The single application using store-and-forward review of lesion images by a dermatologist had a sensitivity of 98% (95% CI 90% to 100%) and specificity of 30% (95% CI 22% to 40%).The number of test failures (lesion images analysed by the applications but classed as 'unevaluable' and excluded by the study authors) ranged from 3 to 31 (or 2% to 18% of lesions analysed). The store-and-forward application had one of the highest rates of test failure (15%). At least one melanoma was classed as unevaluable in three of the four application evaluations. AUTHORS' CONCLUSIONS Smartphone applications using artificial intelligence-based analysis have not yet demonstrated sufficient promise in terms of accuracy, and they are associated with a high likelihood of missing melanomas. Applications based on store-and-forward images could have a potential role in the timely presentation of people with potentially malignant lesions by facilitating active self-management health practices and early engagement of those with suspicious skin lesions; however, they may incur a significant increase in resource and workload. Given the paucity of evidence and low methodological quality of existing studies, it is not possible to draw any implications for practice. Nevertheless, this is a rapidly advancing field, and new and better applications with robust reporting of studies could change these conclusions substantially.
Collapse
Affiliation(s)
- Naomi Chuchu
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Yemisi Takwoingi
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Jacqueline Dinnes
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Rubeta N Matin
- Churchill HospitalDepartment of DermatologyOld RoadHeadingtonOxfordUKOX3 7LE
| | - Oliver Bassett
- Addenbrooke's HospitalPlastic SurgeryHills RoadCambridgeUKCB2 0QQ
| | - Jacqueline F Moreau
- University of Pittsburgh Medical CenterInternal MedicineDepartment of Medicine, Office of EducationUPMC Montefiore Hospital, N715PittsburghUSAPA, 15213
| | - Susan E Bayliss
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Clare Davenport
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
| | - Kathie Godfrey
- The University of Nottinghamc/o Cochrane Skin GroupNottinghamUK
| | - Susan O'Connell
- Cardiff and Vale University Health BoardCEDAR Healthcare Technology Research CentreCardiff Medicentre, University Hospital of Wales, Heath Park CampusCardiffWalesUKCF144UJ
| | - Abhilash Jain
- Imperial College Healthcare NHS trust, St Mary’s HospitalDepartment of Plastic and Reconstructive SurgeryLondonUKW2 1NY
| | - Fiona M Walter
- University of CambridgePublic Health & Primary CareStrangeways Research Laboratory, Worts CausewayCambridgeUKCB1 8RN
| | - Jonathan J Deeks
- University of BirminghamInstitute of Applied Health ResearchBirminghamUKB15 2TT
- University Hospitals Birmingham NHS Foundation Trust and University of BirminghamNIHR Birmingham Biomedical Research CentreBirminghamUK
| | - Hywel C Williams
- University of NottinghamCentre of Evidence Based DermatologyQueen's Medical CentreDerby RoadNottinghamUKNG7 2UH
| | | | | |
Collapse
|
48
|
Selby PJ, Banks RE, Gregory W, Hewison J, Rosenberg W, Altman DG, Deeks JJ, McCabe C, Parkes J, Sturgeon C, Thompson D, Twiddy M, Bestall J, Bedlington J, Hale T, Dinnes J, Jones M, Lewington A, Messenger MP, Napp V, Sitch A, Tanwar S, Vasudev NS, Baxter P, Bell S, Cairns DA, Calder N, Corrigan N, Del Galdo F, Heudtlass P, Hornigold N, Hulme C, Hutchinson M, Lippiatt C, Livingstone T, Longo R, Potton M, Roberts S, Sim S, Trainor S, Welberry Smith M, Neuberger J, Thorburn D, Richardson P, Christie J, Sheerin N, McKane W, Gibbs P, Edwards A, Soomro N, Adeyoju A, Stewart GD, Hrouda D. Methods for the evaluation of biomarkers in patients with kidney and liver diseases: multicentre research programme including ELUCIDATE RCT. Programme Grants Appl Res 2018. [DOI: 10.3310/pgfar06030] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/11/2022]
Abstract
BackgroundProtein biomarkers with associations with the activity and outcomes of diseases are being identified by modern proteomic technologies. They may be simple, accessible, cheap and safe tests that can inform diagnosis, prognosis, treatment selection, monitoring of disease activity and therapy and may substitute for complex, invasive and expensive tests. However, their potential is not yet being realised.Design and methodsThe study consisted of three workstreams to create a framework for research: workstream 1, methodology – to define current practice and explore methodology innovations for biomarkers for monitoring disease; workstream 2, clinical translation – to create a framework of research practice, high-quality samples and related clinical data to evaluate the validity and clinical utility of protein biomarkers; and workstream 3, the ELF to Uncover Cirrhosis as an Indication for Diagnosis and Action for Treatable Event (ELUCIDATE) randomised controlled trial (RCT) – an exemplar RCT of an established test, the ADVIA Centaur® Enhanced Liver Fibrosis (ELF) test (Siemens Healthcare Diagnostics Ltd, Camberley, UK) [consisting of a panel of three markers – (1) serum hyaluronic acid, (2) amino-terminal propeptide of type III procollagen and (3) tissue inhibitor of metalloproteinase 1], for liver cirrhosis to determine its impact on diagnostic timing and the management of cirrhosis and the process of care and improving outcomes.ResultsThe methodology workstream evaluated the quality of recommendations for using prostate-specific antigen to monitor patients, systematically reviewed RCTs of monitoring strategies and reviewed the monitoring biomarker literature and how monitoring can have an impact on outcomes. Simulation studies were conducted to evaluate monitoring and improve the merits of health care. The monitoring biomarker literature is modest and robust conclusions are infrequent. We recommend improvements in research practice. Patients strongly endorsed the need for robust and conclusive research in this area. The clinical translation workstream focused on analytical and clinical validity. Cohorts were established for renal cell carcinoma (RCC) and renal transplantation (RT), with samples and patient data from multiple centres, as a rapid-access resource to evaluate the validity of biomarkers. Candidate biomarkers for RCC and RT were identified from the literature and their quality was evaluated and selected biomarkers were prioritised. The duration of follow-up was a limitation but biomarkers were identified that may be taken forward for clinical utility. In the third workstream, the ELUCIDATE trial registered 1303 patients and randomised 878 patients out of a target of 1000. The trial started late and recruited slowly initially but ultimately recruited with good statistical power to answer the key questions. ELF monitoring altered the patient process of care and may show benefits from the early introduction of interventions with further follow-up. The ELUCIDATE trial was an ‘exemplar’ trial that has demonstrated the challenges of evaluating biomarker strategies in ‘end-to-end’ RCTs and will inform future study designs.ConclusionsThe limitations in the programme were principally that, during the collection and curation of the cohorts of patients with RCC and RT, the pace of discovery of new biomarkers in commercial and non-commercial research was slower than anticipated and so conclusive evaluations using the cohorts are few; however, access to the cohorts will be sustained for future new biomarkers. The ELUCIDATE trial was slow to start and recruit to, with a late surge of recruitment, and so final conclusions about the impact of the ELF test on long-term outcomes await further follow-up. The findings from the three workstreams were used to synthesise a strategy and framework for future biomarker evaluations incorporating innovations in study design, health economics and health informatics.Trial registrationCurrent Controlled Trials ISRCTN74815110, UKCRN ID 9954 and UKCRN ID 11930.FundingThis project was funded by the NIHR Programme Grants for Applied Research programme and will be published in full inProgramme Grants for Applied Research; Vol. 6, No. 3. See the NIHR Journals Library website for further project information.
Collapse
Affiliation(s)
- Peter J Selby
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - Rosamonde E Banks
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
| | - Walter Gregory
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - Jenny Hewison
- Leeds Institute of Health Sciences, University of Leeds, Leeds, UK
| | - William Rosenberg
- Institute for Liver and Digestive Health, Division of Medicine, University College London, London, UK
| | - Douglas G Altman
- Centre for Statistics in Medicine, University of Oxford, Oxford, UK
| | - Jonathan J Deeks
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Christopher McCabe
- Department of Emergency Medicine, University of Alberta Hospital, Edmonton, AB, Canada
| | - Julie Parkes
- Primary Care and Population Sciences Academic Unit, University of Southampton, Southampton, UK
| | | | | | - Maureen Twiddy
- Leeds Institute of Health Sciences, University of Leeds, Leeds, UK
| | - Janine Bestall
- Leeds Institute of Health Sciences, University of Leeds, Leeds, UK
| | | | - Tilly Hale
- LIVErNORTH Liver Patient Support, Newcastle upon Tyne, UK
| | - Jacqueline Dinnes
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Marc Jones
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | | | | | - Vicky Napp
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - Alice Sitch
- Institute of Applied Health Research, University of Birmingham, Birmingham, UK
| | - Sudeep Tanwar
- Institute for Liver and Digestive Health, Division of Medicine, University College London, London, UK
| | - Naveen S Vasudev
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - Paul Baxter
- Leeds Institute of Cardiovascular and Metabolic Medicine, University of Leeds, Leeds, UK
| | - Sue Bell
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - David A Cairns
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
| | | | - Neil Corrigan
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - Francesco Del Galdo
- Leeds Institute of Rheumatic and Musculoskeletal Medicine, University of Leeds, Leeds, UK
| | - Peter Heudtlass
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - Nick Hornigold
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
| | - Claire Hulme
- Leeds Institute of Health Sciences, University of Leeds, Leeds, UK
| | - Michelle Hutchinson
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
| | - Carys Lippiatt
- Department of Specialist Laboratory Medicine, Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | | | - Roberta Longo
- Leeds Institute of Health Sciences, University of Leeds, Leeds, UK
| | - Matthew Potton
- Leeds Institute of Clinical Trials Research, University of Leeds, Leeds, UK
| | - Stephanie Roberts
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
| | - Sheryl Sim
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
| | - Sebastian Trainor
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
| | - Matthew Welberry Smith
- Clinical and Biomedical Proteomics Group, Leeds Institute of Cancer and Pathology, University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | - James Neuberger
- University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK
| | | | - Paul Richardson
- Royal Liverpool and Broadgreen University Hospitals NHS Trust, Liverpool, UK
| | - John Christie
- Royal Devon and Exeter NHS Foundation Trust, Exeter, UK
| | - Neil Sheerin
- Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
| | - William McKane
- Sheffield Teaching Hospitals NHS Foundation Trust, Sheffield, UK
| | - Paul Gibbs
- Portsmouth Hospitals NHS Trust, Portsmouth, UK
| | | | - Naeem Soomro
- Newcastle upon Tyne Hospitals NHS Foundation Trust, Newcastle upon Tyne, UK
| | | | - Grant D Stewart
- NHS Lothian, Edinburgh, UK
- Academic Urology Group, University of Cambridge, Cambridge, UK
| | - David Hrouda
- Charing Cross Hospital, Imperial College Healthcare NHS Trust, London, UK
| |
Collapse
|
49
|
Dinnes J, Saleh D, Newton-Bishop J, Cheung ST, Nathan P, Matin RN, Chuchu N, Bayliss SE, Takwoingi Y, Davenport C, Godfrey K, O'Sullivan C, Deeks JJ, Williams HC. Tests to assist in the staging of cutaneous melanoma: a generic protocol. Hippokratia 2017. [DOI: 10.1002/14651858.cd012806] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
Affiliation(s)
- Jacqueline Dinnes
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Daniel Saleh
- Newcastle Hospitals NHS Trust, Royal Victoria Infirmary; Newcastle Hospitals; Newcastle UK
| | - Julia Newton-Bishop
- University of Leeds; Section of Epidemiology and Biostatistics; St James's Hospital Leeds UK LS9 7TF
| | - Seau Tak Cheung
- Dudley Hospitals Foundation Trust, Corbett Hospital; Department of Dermatology; Wicarage Road Stourbridge UK DY8 4JB
| | - Paul Nathan
- Mount Vernon Hospital; Mount Vernon Cancer Centre; Rickmansworth Road Northwood UK HA6 2RN
| | - Rubeta N Matin
- Churchill Hospital; Department of Dermatology; Old Road Headington Oxford UK OX3 7LJ
| | - Naomi Chuchu
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Susan E Bayliss
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Yemisi Takwoingi
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Clare Davenport
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Kathie Godfrey
- The University of Nottingham; c/o Cochrane Skin Group; Nottingham UK
| | | | - Jonathan J Deeks
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Hywel C Williams
- The University of Nottingham; Centre of Evidence Based Dermatology; Queen's Medical Centre Derby Road Nottingham UK NG7 2UH
| |
Collapse
|
50
|
Dinnes J, Matin RN, Webster AC, Lawton P, Chuchu N, Bayliss SE, Takwoingi Y, Davenport C, Godfrey K, O'Sullivan C, Deeks JJ, Williams HC. Tests to assist in the staging of cutaneous squamous cell carcinoma: a generic protocol. Cochrane Database Syst Rev 2017. [DOI: 10.1002/14651858.cd012773] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Affiliation(s)
- Jacqueline Dinnes
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Rubeta N Matin
- Churchill Hospital; Department of Dermatology; Old Road Headington Oxford UK OX3 7LJ
| | - Angela C Webster
- The University of Sydney; Sydney School of Public Health; Edward Ford Building A27 Sydney NSW Australia 2006
| | - Pat Lawton
- Nottingham University Hospitals NHS Trust; Department of Oncology; City Hospital Campus Nottingham City Hospital NHS Trust Campus, Hucknall Road Nottingham UK NG5 1PB
| | - Naomi Chuchu
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Susan E Bayliss
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Yemisi Takwoingi
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Clare Davenport
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Kathie Godfrey
- The University of Nottingham; c/o Cochrane Skin Group; Nottingham UK
| | | | - Jonathan J Deeks
- University of Birmingham; Institute of Applied Health Research; Birmingham UK B15 2TT
| | - Hywel C Williams
- The University of Nottingham; Centre of Evidence Based Dermatology; Queen's Medical Centre Derby Road Nottingham UK NG7 2UH
| |
Collapse
|