1
|
Crowley RJ, Tan YJ, Ioannidis JPA. Empirical assessment of bias in machine learning diagnostic test accuracy studies. J Am Med Inform Assoc 2020; 27:1092-1101. [PMID: 32548642 PMCID: PMC7647361 DOI: 10.1093/jamia/ocaa075] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2020] [Revised: 04/12/2020] [Accepted: 04/24/2020] [Indexed: 12/29/2022] Open
Abstract
OBJECTIVE Machine learning (ML) diagnostic tools have significant potential to improve health care. However, methodological pitfalls may affect diagnostic test accuracy studies used to appraise such tools. We aimed to evaluate the prevalence and reporting of design characteristics within the literature. Further, we sought to empirically assess whether design features may be associated with different estimates of diagnostic accuracy. MATERIALS AND METHODS We systematically retrieved 2 × 2 tables (n = 281) describing the performance of ML diagnostic tools, derived from 114 publications in 38 meta-analyses, from PubMed. Data extracted included test performance, sample sizes, and design features. A mixed-effects metaregression was run to quantify the association between design features and diagnostic accuracy. RESULTS Participant ethnicity and blinding in test interpretation was unreported in 90% and 60% of studies, respectively. Reporting was occasionally lacking for rudimentary characteristics such as study design (28% unreported). Internal validation without appropriate safeguards was used in 44% of studies. Several design features were associated with larger estimates of accuracy, including having unreported (relative diagnostic odds ratio [RDOR], 2.11; 95% confidence interval [CI], 1.43-3.1) or case-control study designs (RDOR, 1.27; 95% CI, 0.97-1.66), and recruiting participants for the index test (RDOR, 1.67; 95% CI, 1.08-2.59). DISCUSSION Significant underreporting of experimental details was present. Study design features may affect estimates of diagnostic performance in the ML diagnostic test accuracy literature. CONCLUSIONS The present study identifies pitfalls that threaten the validity, generalizability, and clinical value of ML diagnostic tools and provides recommendations for improvement.
Collapse
Affiliation(s)
- Ryan J Crowley
- Meta-Research Innovation Center at Stanford, Stanford University, Stanford, California, USA
- Department of Bioengineering, Stanford School of Engineering, Stanford University, Stanford, California, USA
| | - Yuan Jin Tan
- Meta-Research Innovation Center at Stanford, Stanford University, Stanford, California, USA
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA
| | - John P A Ioannidis
- Meta-Research Innovation Center at Stanford, Stanford University, Stanford, California, USA
- Department of Epidemiology and Population Health, Stanford University School of Medicine, Stanford, California, USA
- Stanford Prevention Research Center, Department of Medicine, Stanford Medicine, Stanford University, Stanford, California, USA
- Department of Biomedical Data Science, Stanford Medicine, Stanford University, Stanford, California, USA
- Department of Statistics, School of Humanities and Science, Stanford University, Stanford, California, USA
| |
Collapse
|
2
|
Alldred SK, Takwoingi Y, Guo B, Pennant M, Deeks JJ, Neilson JP, Alfirevic Z. First trimester serum tests for Down's syndrome screening. Cochrane Database Syst Rev 2015; 2015:CD011975. [PMID: 26617074 PMCID: PMC6465076 DOI: 10.1002/14651858.cd011975] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
BACKGROUND Down's syndrome occurs when a person has three, rather than two copies of chromosome 21; or the specific area of chromosome 21 implicated in causing Down's syndrome. It is the commonest congenital cause of mental disability and also leads to numerous metabolic and structural problems. It can be life-threatening, or lead to considerable ill health, although some individuals have only mild problems and can lead relatively normal lives. Having a baby with Down's syndrome is likely to have a significant impact on family life.Noninvasive screening based on biochemical analysis of maternal serum or urine, or fetal ultrasound measurements, allows estimates of the risk of a pregnancy being affected and provides information to guide decisions about definitive testing. However, no test can predict the severity of problems a person with Down's syndrome will have. OBJECTIVES The aim of this review was to estimate and compare the accuracy of first trimester serum markers for the detection of Down's syndrome in the antenatal period, both as individual markers and as combinations of markers. Accuracy is described by the proportion of fetuses with Down's syndrome detected by screening before birth (sensitivity or detection rate) and the proportion of women with a low risk (normal) screening test result who subsequently had a baby unaffected by Down's syndrome (specificity). SEARCH METHODS We conducted a sensitive and comprehensive literature search of MEDLINE (1980 to 25 August 2011), Embase (1980 to 25 August 2011), BIOSIS via EDINA (1985 to 25 August 2011), CINAHL via OVID (1982 to 25 August 2011), The Database of Abstracts of Reviews of Effectiveness (The Cochrane Library 25 August 2011), MEDION (25 August 2011), The Database of Systematic Reviews and Meta-Analyses in Laboratory Medicine (25 August 2011), The National Research Register (Archived 2007), Health Services Research Projects in Progress database (25 August 2011). We did forward citation searching ISI citation indices, Google Scholar and PubMed 'related articles'. We did not apply a diagnostic test search filter. We also searched reference lists and published review articles. SELECTION CRITERIA We included studies in which all women from a given population had one or more index test(s) compared to a reference standard (either chromosomal verification or macroscopic postnatal inspection). Both consecutive series and diagnostic case-control study designs were included. Randomised trials where individuals were randomised to different screening strategies and all verified using a reference standard were also eligible for inclusion. Studies in which test strategies were compared head-to-head either in the same women, or between randomised groups were identified for inclusion in separate comparisons of test strategies. We excluded studies if they included less than five Down's syndrome cases, or more than 20% of participants were not followed up. DATA COLLECTION AND ANALYSIS We extracted data as test positive or test negative results for Down's and non-Down's pregnancies allowing estimation of detection rates (sensitivity) and false positive rates (1-specificity). We performed quality assessment according to QUADAS (Quality Assessment of Diagnostic Accuracy Studies) criteria. We used hierarchical summary ROC meta-analytical methods or random-effects logistic regression methods to analyse test performance and compare test accuracy as appropriate. Analyses of studies allowing direct and indirect comparisons between tests were undertaken. MAIN RESULTS We included 56 studies (reported in 68 publications) involving 204,759 pregnancies (including 2113 with Down's syndrome). Studies were generally of good quality, although differential verification was common with invasive testing of only high-risk pregnancies. We evaluated 78 test combinations formed from combinations of 18 different tests, with or without maternal age; ADAM12 (a disintegrin and metalloprotease), AFP (alpha-fetoprotein), inhibin, PAPP-A (pregnancy-associated plasma protein A, ITA (invasive trophoblast antigen), free βhCG (beta human chorionic gonadotrophin), PlGF (placental growth factor), SP1 (Schwangerschafts protein 1), total hCG, progesterone, uE3 (unconjugated oestriol), GHBP (growth hormone binding protein), PGH (placental growth hormone), hyperglycosylated hCG, ProMBP (proform of eosinophil major basic protein), hPL (human placental lactogen), (free αhCG, and free ßhCG to AFP ratio. Direct comparisons between two or more tests were made in 27 studies.Meta-analysis of the nine best performing or frequently evaluated test combinations showed that a test strategy involving maternal age and a double marker combination of PAPP-A and free ßhCG significantly outperformed the individual markers (with or without maternal age) detecting about seven out of every 10 Down's syndrome pregnancies at a 5% false positive rate (FPR). Limited evidence suggested that marker combinations involving PAPP-A may be more sensitive than those without PAPP-A. AUTHORS' CONCLUSIONS Tests involving two markers in combination with maternal age, specifically PAPP-A, free βhCG and maternal age are significantly better than those involving single markers with and without age. They detect seven out of 10 Down's affected pregnancies for a fixed 5% FPR. The addition of further markers (triple tests) has not been shown to be statistically superior; the studies included are small with limited power to detect a difference.The screening blood tests themselves have no adverse effects for the woman, over and above the risks of a routine blood test. However some women who have a 'high risk' screening test result, and are given amniocentesis or chorionic villus sampling (CVS) have a risk of miscarrying a baby unaffected by Down's. Parents will need to weigh up this risk when deciding whether or not to have an amniocentesis or CVS following a 'high risk' screening test result.
Collapse
Affiliation(s)
- S Kate Alldred
- The University of LiverpoolDepartment of Women's and Children's HealthFirst Floor, Liverpool Women's NHS Foundation TrustCrown StreetLiverpoolUKL8 7SS
| | - Yemisi Takwoingi
- University of BirminghamPublic Health, Epidemiology and BiostatisticsEdgbastonBirminghamUKB15 2TT
| | - Boliang Guo
- University of NottinghamSchool of MedicineCLAHRC, C floor, IHM, Jubilee CampusUniversity of Nottingham, Triumph RoadNottinghamEast MidlandsUKNG7 2TU
| | - Mary Pennant
- Cambridgeshire County CouncilPublic Health DirectorateCambridgeUK
| | - Jonathan J Deeks
- University of BirminghamPublic Health, Epidemiology and BiostatisticsEdgbastonBirminghamUKB15 2TT
| | - James P Neilson
- The University of LiverpoolDepartment of Women's and Children's HealthFirst Floor, Liverpool Women's NHS Foundation TrustCrown StreetLiverpoolUKL8 7SS
| | - Zarko Alfirevic
- The University of LiverpoolDepartment of Women's and Children's HealthFirst Floor, Liverpool Women's NHS Foundation TrustCrown StreetLiverpoolUKL8 7SS
| | | |
Collapse
|
3
|
Krantz D, Hallahan T, Ravens R, He K, Cuckle H, Sherwin J, Carmichael J. First trimester Down syndrome screening with dried blood spots using a dual analyte free beta hCG and PAPP-A immunofluorometric assay. Prenat Diagn 2011; 31:869-74. [PMID: 21692092 DOI: 10.1002/pd.2792] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2011] [Revised: 04/07/2011] [Accepted: 04/29/2011] [Indexed: 11/07/2022]
Abstract
OBJECTIVE To determine the effectiveness of first trimester Down syndrome screening with dried blood spots using a dual analyte free beta human chorionic gonadotrophin (hCG)/pregnancy-associated plasma protein A (PAPP-A) immunofluorometric assay. METHOD An initial retrospective study of 54 Down syndrome cases and 1064 control specimens was performed followed by a series of 146,513 specimens from routine screening. Detection rates at a fixed 5% false-positive rate were determined separately based on reference data from the retrospective study set and then adjusted based on the routine screening study set. RESULTS On the basis of the retrospective analysis, the estimated detection rate using free beta hCG, PAPP-A and maternal age varied from 78% at 9 weeks of pregnancy to 70% at 13 weeks of pregnancy. Using a combined protocol, including NT, the detection rate varied from 92 to 90% between 9 and 13 weeks of gestation. Adjusting distribution parameters based on the routine screening dataset reduced the detection rate by at most 1%. CONCLUSION Analysis of free beta hCG and PAPP-A using a dual analyte dried blood spot assay is an effective tool in Down syndrome screening, adding an important option for those considering implementation or modification of existing prenatal screening programs.
Collapse
Affiliation(s)
- David Krantz
- NTD Laboratories/PerkinElmer, Melville, NY 11747, USA.
| | | | | | | | | | | | | |
Collapse
|
4
|
Ramos-Corpas DJ, Santiago JC. Combined test + inhibin A at week 13 in contingent sequential testing: an interesting alternative for first-trimester prenatal screening for Down Syndrome. Prenat Diagn 2008; 28:833-8. [DOI: 10.1002/pd.2063] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
5
|
Evans MI, Krantz DA, Hallahan TW, Galen RS. Meta-analysis of first trimester Down syndrome screening studies: free beta-human chorionic gonadotropin significantly outperforms intact human chorionic gonadotropin in a multimarker protocol. Am J Obstet Gynecol 2007; 196:198-205. [PMID: 17346522 DOI: 10.1016/j.ajog.2006.08.001] [Citation(s) in RCA: 23] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2006] [Revised: 05/18/2006] [Accepted: 08/01/2006] [Indexed: 10/23/2022]
Abstract
OBJECTIVE The purpose of this study was to compare free beta and intact human chorionic gonadotropin in first trimester screening with pregnancy-associated plasma protein-A and nuchal translucency. STUDY DESIGN A Monte Carlo simulation trial was conducted based on a literature review of the PUBMED database (1966 to November 2005). RESULTS In younger patients (< 35 years), detection of Down syndrome increased by 4, 5, 6, and 7 percentage points when free beta was added to pregnancy-associated plasma protein-A and nuchal translucency compared with 0, 0, 2, and 4 percentage points for intact human chorionic gonadotropin at 9-12 weeks' gestation, respectively. In advanced maternal age patients (> or = 35), inclusion of free beta-human chorionic gonadotropin reduced the false-positive rate by 2.5, 3.1, 3.8, and 4.4 percentage points compared with 0.1, 0.3, 1.0, and 2.2 percentage points for intact human chorionic gonadotropin at 9-12 weeks, respectively. CONCLUSION The results of our analysis suggest that in a first-trimester Down syndrome screening protocol free beta-human chorionic gonadotropin achieves higher sensitivity and lower false-positive results than intact human chorionic gonadotropin . Moreover, intact human chorionic gonadotropin does not add substantially to screening performance until the end of the first trimester.
Collapse
Affiliation(s)
- Mark I Evans
- Comprehensive Genetics, and Mt Sinai School of Medicine, New York, NY 10021, USA.
| | | | | | | |
Collapse
|
6
|
Thorlacius LS, Blakney G, Krahn J, Bamforth F, Higgins TN. Biochemistry testing associated with pregnancy and the newborn period -- a lot has changed since you were a baby! Clin Biochem 2006; 39:519-41. [PMID: 16730256 DOI: 10.1016/j.clinbiochem.2006.03.008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/22/2005] [Revised: 03/13/2006] [Accepted: 03/14/2006] [Indexed: 10/24/2022]
Abstract
Everyone has been a newborn, and everyone's mother has been pregnant. Despite the commonality of these events, medical care and the clinical chemistry laboratory's role in it have changed remarkably over the last 50 years. This review is a historical overview of clinical chemistry testing that is related to pregnancy and the newborn period.
Collapse
|
7
|
Weisz B, Rodeck CH. An update on antenatal screening for Down's syndrome and specific implications for assisted reproduction pregnancies. Hum Reprod Update 2006; 12:513-8. [PMID: 16672245 DOI: 10.1093/humupd/dml021] [Citation(s) in RCA: 15] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/14/2022] Open
Abstract
Since the introduction of antenatal serum screening for Down's syndrome (DS) more than two decades ago, several screening approaches have been utilized in routine clinical practice. The current DS screening strategies involve mid-trimester serum biochemistry tests, first trimester tests combining sonographic markers and serum biochemistry and integration of first and second trimester markers. In this review, we evaluate the performance of DS screening strategies according to the Serum, Urine and Ultrasound Screening Study (SURUSS), the First and Second Trimester Evaluation of Risks (FASTER) Trial and the Serum Biochemistry and Fetal Nuchal Translucency Screening (BUN) Study. We also evaluate the performance of first trimester screening in studies and meta-analyses by other groups. Specific issues related to assisted reproduction technology (ART) pregnancies are also addressed in this review.
Collapse
Affiliation(s)
- Boaz Weisz
- Department of Obstetrics and Gynaecology, Sheba Medical Center, Tel-Hashomer, Israel.
| | | |
Collapse
|
8
|
Wøjdemann KR, Shalmi AC, Christiansen M, Larsen SO, Sundberg K, Brocks V, Bang J, Nørgaard-Pedersen B, Tabor A. Improved first-trimester Down syndrome screening performance by lowering the false-positive rate: a prospective study of 9941 low-risk women. ULTRASOUND IN OBSTETRICS & GYNECOLOGY : THE OFFICIAL JOURNAL OF THE INTERNATIONAL SOCIETY OF ULTRASOUND IN OBSTETRICS AND GYNECOLOGY 2005; 25:227-233. [PMID: 15736185 DOI: 10.1002/uog.1834] [Citation(s) in RCA: 37] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
OBJECTIVE To determine the performance of screening for Down syndrome (DS) and other major chromosomal abnormalities using nuchal translucency (NT), free beta-human chorionic gonadotropin (beta-hCG) and pregnancy-associated plasma protein-A (PAPP-A) in a prospective study of a non-selected population. METHODS Of 9941 women with an early ultrasound examination, NT was measured in 8622 singleton pregnancies with a gestational age between 10 + 3 and 13 + 6 weeks. beta-hCG and PAPP-A were analyzed in 6441 cases. Detection rates (DR) and false-positive rates (FPR) for the NT screening, the double test (beta-hCG and PAPP-A) and the combined test (NT and the double test) were calculated using a 1 : 250 cut-off. RESULTS NT could be measured in 97.5% of cases. The DR for DS with NT screening alone was 75% with a FPR of only 1.8%. The double test detected 73% and the combined test 91%, for FPRs of 8.8% and 2.1%, respectively. We detected 80% of fetuses with other major chromosomal abnormalities with a combination of NT screening and other ultrasound findings. Low beta-hCG and PAPP-A values (below 0.4 MoM) were observed in 0.5% of the women including all cases of triploidy and trisomy 18 and 13. CONCLUSIONS The performance of a screening strategy for DS using a combination of NT and the double test was superior to that using either NT or the double test alone due to a very low FPR and a higher DR.
Collapse
Affiliation(s)
- K R Wøjdemann
- Copenhagen University Hospital, Department of Obstetrics and Gynaecology, Hvidovre Hospital, Copenhagen, Denmark.
| | | | | | | | | | | | | | | | | |
Collapse
|
9
|
Babbur V, Lees CC, Goodburn SF, Morris N, Breeze ACG, Hackett GA. Prospective audit of a one-centre combined nuchal translucency and triple test programme for the detection of trisomy 21. Prenat Diagn 2005; 25:465-9. [PMID: 15966036 DOI: 10.1002/pd.1163] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
OBJECTIVES To determine detection and false-positive rates for trisomy 21 using two-stage combined nuchal translucency (NT) and triple testing, whilst disclosing abnormal nuchal measurements at the scan. METHODS A prospective audit in a UK women's hospital, of 3188 women with singleton pregnancies, requesting screening for trisomy 21. Median age was 37 years (range 19-46). Women were offered NT screening at 11 to 14 weeks. Those with NT > or =3 mm were offered chorionic villus sampling. Those declining CVS, and those with NT <3 mm, were offered early triple tests. Women with a term combined risk of trisomy 21 > or = 1:250, based on age, NT, and triple test results were offered amniocentesis. RESULTS Using a 3-mm NT 'cut-off' identified 16/25 cases of trisomy 21 (64%; 95% CI 38.8, 78.9). Of 2725 women who had a combined nuchal plus triple test assessment, 79 (2.6%) had a > or = 1:250 term risk of trisomy 21. Forty (1.3%) had amniocentesis identifying 6/9 remaining cases (67%:95% CI:27.9, 92.5). Overall, the detection rate was 88% (95% CI:68.8, 97.5) for a 4.8% FPR. For the screened population, to achieve an 88% detection rate using the triple test alone, the predicted FPR would be 20%. Conversely, for an FPR of 4.8% using the triple test alone, the detection rate would be only 60%. CONCLUSION In a high-risk group, the combination of NT with triple test offers detection of trisomy 21 at least equivalent to either test, while allowing disclosure of an abnormal NT at the scan and reducing the FPR. Importantly, the FPR is less than 5%, considerably lower than expected for triple test alone for this population.
Collapse
|
10
|
Stenhouse EJ, Crossley JA, Aitken DA, Brogan K, Cameron AD, Connor JM. First-trimester combined ultrasound and biochemical screening for Down syndrome in routine clinical practice. Prenat Diagn 2004; 24:774-80. [PMID: 15503268 DOI: 10.1002/pd.980] [Citation(s) in RCA: 43] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/11/2022]
Abstract
OBJECTIVES To assess the effectiveness of combined ultrasound and biochemical (CUB) screening for chromosome abnormalities in singleton pregnancies in a routine antenatal clinic and laboratory setting. METHODS Women whose pregnancies fell within the gestational age range of 11 to 14 weeks by ultrasound assessment were offered CUB screening on the basis of measurement of nuchal translucency (NT), maternal serum free beta-human chorionic gonadotrophin (FbetahCG) and pregnancy-associated plasma protein A (PAPP-A). NT measurements were obtained using a standardised method defined by the Fetal Medicine Foundation and FbetahCG, and PAPP-A were measured using the DELFIA immunoassay system. Each screening marker measurement was converted to a multiple of the appropriate gestational median and a risk was derived using previously published parameters for each marker in chromosomally abnormal and unaffected pregnancies. A combined risk of Down syndrome and of trisomy 18/13, incorporating the maternal age risk, was calculated for all women. Invasive diagnostic testing was offered to women whose combined risk exceeded the cut-off risk of 1 in 250 (term). RESULTS Five thousand and eighty-four women accepted a first-trimester screening test for Down syndrome, representing 75% of the eligible booking population. Out of the population eligible for CUB screening at the time of booking, NT measurements were obtained from 93% at the first clinic visit and 7% had to return for a second attempt. After excluding women who defaulted on a return visit, satisfactory NT measurements were obtained in 99.5% of pregnancies. Fifteen cases of Down syndrome and eleven pregnancies with other chromosome abnormalities were ascertained. The detection rate for Down syndrome was 93% (14/15) at a false-positive rate of 5.9% and for all chromosome abnormalities it was 96% (25/26) at an overall false-positive rate of 6.3%. CONCLUSIONS CUB screening offers a significant improvement in sensitivity over second-trimester biochemical screening and is deliverable within a routine prenatal clinical setting.
Collapse
Affiliation(s)
- E J Stenhouse
- Fetal Medicine Department, Queen Mother's Maternity Hospital, Yorkhill Hospitals, Glasgow, UK
| | | | | | | | | | | |
Collapse
|
11
|
Abstract
Historically, obstetrician/gynaecologists have utilized screening procedures as a major component of their routine clinical practice. The Papanicolaou (Pap) smear technique for cervical cancer is more than 50 years old and has saved countless lives by identifying who among the supposedly low-risk group of patients is in fact at high risk of cervical cancer. The introduction of alpha-fetoprotein screening for neural tube defects in the 1970s, low alpha-fetoprotein level for Down syndrome in the 80s, multiple markers in the 90s and now, after the millennium combined ultrasound and biochemistry for a more precise risk identification, has radically changed our approach towards optimizing the sensitivity, specificity and positive and negative predictive values of screening. As these techniques improve, old mainstays of attributable risks such as advanced maternal age will probably fade into oblivion as stand-alone criteria. The specifics of screening techniques will vary by disorder, but the general principles are important and can be applied to all new technologies as they come on line. In fact, the real 'take-home message' is that one must use these principles when looking at new technologies to understand their role in emerging practice.
Collapse
Affiliation(s)
- Mark I Evans
- Columbia University College of Physicians and Surgeons, St Luke's Roosevelt Hospital Center, 1000 10th Avenue, New York, NY 10019, USA
| | | | | |
Collapse
|
12
|
Crossley JA, Aitken DA, Cameron AD, McBride E, Connor JM. Combined ultrasound and biochemical screening for Down's syndrome in the first trimester: a Scottish multicentre study. BJOG 2002; 109:667-76. [PMID: 12118646 DOI: 10.1111/j.1471-0528.2002.01394.x] [Citation(s) in RCA: 95] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
OBJECTIVE To evaluate the use of ultrasound measurements of fetal nuchal translucency (NT) obtained in a routine antenatal clinic setting in combination with appropriate biochemical markers as a first trimester screening test for Down's Syndrome. DESIGN Multicentre observational study. SETTING Fifteen Scottish maternity units. POPULATION Pregnant women (n = 17,229) attending routine antenatal clinics at 10-14 weeks of gestation. METHODS NT measurements were attempted in all women along with the measurement of maternal serum free beta human chorionic gonadotrophin (F beta hCG) and pregnancy-associated plasma protein-A (PAPP-A). All results were converted to multiples of the appropriate gestational median (MoM) and using a statistical model the risk of an affected pregnancy was derived. No results were given to participating women but all were offered routine second trimester biochemical screening. All cases of Down's Syndrome within the study group were ascertained and the detection rate for each marker was estimated. MAIN OUTCOME MEASURES Success rate of obtaining NT measurements and overall effectiveness of ultrasound and biochemical markers individually and in combination for the detection of Down's Syndrome pregnancies. RESULTS NT measurements were obtained in 72.9% of women and blood samples in 98.4%. Forty-five cases of Down's Syndrome were ascertained (2.6/1,000). NT measurements were obtained in 37 cases (median NT 1.65 MoM), blood samples in 42 cases and both NT and blood in 34 cases. In combination with the a priori maternal age risk, observed detection rates at a 5% false positive rate were 20/37 (54%) for NT, 23/42 (55%) for F beta hCG and PAPP-A and 28/34 (82%) for a combination of NT, F beta hCG and PAPP-A using a cutoff risk of 1:250. The effect of failing to obtain NT measurements in all cases reduces the overall detection rate to 62% (i.e. 28/45) if the entire series of affected pregnancies within the study group is considered. CONCLUSIONS NT in combination with appropriate serum markers has the potential to detect over 80% of Down's Syndrome fetuses in early pregnancy. However, NT measurement is highly operator-dependent. It requires training, external quality control and adequate time to allow accurate measurement, otherwise suboptimal performance will result.
Collapse
|
13
|
Ochshorn Y, Kupferminc MJ, Wolman I, Orr-Urtreger A, Jaffa AJ, Yaron Y. First trimester PAPP-A in the detection of non-Down syndrome aneuploidy. Prenat Diagn 2001; 21:547-9. [PMID: 11494288 DOI: 10.1002/pd.105] [Citation(s) in RCA: 14] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
Abstract
Combined first trimester screening using pregnancy associated plasma protein-A (PAPP-A), free beta-human chorionic gonadotrophin, and nuchal translucency (NT), is currently accepted as probably the best combination for the detection of Down syndrome (DS). Current first trimester algorithms provide computed risks only for DS. However, low PAPP-A is also associated with other chromosome anomalies such as trisomy 13, 18, and sex chromosome aneuploidy. Thus, using currently available algorithms, some chromosome anomalies may not be detected. The purpose of the present study was to establish a low-end cut-off value for PAPP-A that would increase the detection rates for non-DS chromosome anomalies. The study included 1408 patients who underwent combined first trimester screening. To determine a low-end cut-off value for PAPP-A, a Receiver-Operator Characteristic (ROC) curve analysis was performed. In the entire study group there were 18 cases of chromosome anomalies (trisomy 21, 13, 18, sex chromosome anomalies), 14 of which were among screen-positive patients, a detection rate of 77.7% for all chromosome anomalies (95% CI: 55.7-99.7%). ROC curve analysis detected a statistically significant cut-off for PAPP-A at 0.25 MoM. If the definition of screen-positive were to also include patients with PAPP-A<0.25 MoM, the detection rate would increase to 88.8% for all chromosome anomalies (95% CI: 71.6-106%). This low cut-off value may be used until specific algorithms are implemented for non-Down syndrome aneuploidy.
Collapse
Affiliation(s)
- Y Ochshorn
- Prenatal Diagnosis Unit, Genetic Institute, Sourasky Medical Center, 6 Weizmann Street, Tel Aviv 64239, Israel
| | | | | | | | | | | |
Collapse
|
14
|
Abstract
Of all markers evaluated for first-trimester biochemical screening for Down syndrome (DS), PAPP-A and free beta-hCG emerged as the most predictive. The combined test uses these markers in conjunction with nuchal translucency measurements, and is estimated to achieve a DS detection rate of 80% to 85% at a 5% false-positive rate. The integrated test, combining first-trimester sonographic and biochemical markers with second-trimester markers, provides a single estimate of a patients DS risk, and may yield a DS detection rate of 94% at a 5% false-positive rate. The acceptability and feasibility of this test, however, remain to be proved.
Collapse
Affiliation(s)
- Y Yaron
- Prenatal Diagnosis Unit, Genetic Institute and Department of Obstetrics and Gynecology, Lis Maternity Hospital, Tel Aviv Sourasky Medical Center, Tel Aviv, Israel.
| | | |
Collapse
|
15
|
Abstract
By 1998, most health authorities offered antenatal screening for Down's syndrome, usually by biochemical methods. To date, the development of this form of screening has not been coordinated by a national body and, consequently, there are wide variations in practice between localities. Fortunately, many of these variations have not led to any noticeable inequality of health provision, but the wide variation in risk cut offs used by different centres does. Other variations merely lead to potentially unnecessary expenditure; whereas it is believed that adding extra tests to the screening procedure is beneficial (such as double test to triple test), statistical evaluation of the confidence intervals for the detection rates quoted indicates that there is no evidence that the extra test provides an increase in detection. The cervical screening programme has progressively improved, partly through the auspices of a national framework. A similar national approach would benefit Down's screening and is only now being considered: the national screening committee (NSC) is currently drafting recommendations. To ensure optimum screening performance, the NSC should specify the risk thresholds applied, the screening protocols to be used--that is, an opt-in programme with a minimum (possibly even a maximum) of two biochemical analytes or a nuchal fold evaluation--and perhaps should even recommend national population parameters to be used for risk calculation. It might even be advisable for statistical work to be carried out to determine whether local derivation of medians is truly necessary. Furthermore, defined options for older women could be specified--for example, should all older patients have the option to proceed directly to amniocentesis if they wish or should National Health Service amniocentesis only be available for those with a "high risk" screening result. The difficulties that will face the NSC in deciding which screening policy to adopt are also considered; specifically, the lack of evidence to suggest that triple testing is superior to double testing, and the lack of evidence to prove the superiority of one analyte over another. This inadequacy of evidence is not from want of trying, but is caused by the problems of collecting enough data to provide statistical significance. Finally, there is one important difference between cervical and Down's syndrome screening that has a major impact on the advice given by any "expert"; namely, patents. Many aspects of Down's screening are subject to patents and, therefore, there is more potential for apparently uncontroversial decisions to rebound with future retrospective patent infringement claims. Thus, it would be sensible to insist that any member of a national body deciding upon Down's screening policy must fully disclose all potential conflicts of interest, both personal and family, before they are allowed to sit on the committee. Furthermore, if a national policy is decided upon, worldwide patent searches should be carried out to determine whether there are any possible unforeseen legal consequences of any recommendation.
Collapse
Affiliation(s)
- T M Reynolds
- Queen's Hospital and Division of Clinical Sciences, Wolverhampton University, Wolverhampton WV1 1SB, UK.
| |
Collapse
|
16
|
Christiansen M, Oxvig C, Wagner JM, Qin QP, Nguyen TH, Overgaard MT, Larsen SO, Sottrup-Jensen L, Gleich GJ, Norgaard-Pedersen B. The proform of eosinophil major basic protein: a new maternal serum marker for Down syndrome. Prenat Diagn 1999; 19:905-10. [PMID: 10521813 DOI: 10.1002/(sici)1097-0223(199910)19:10<905::aid-pd658>3.0.co;2-q] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
The proform of eosinophil major basic protein (proMBP), the most abundant protein in the eosinophil specific granule, is synthesized by the placenta and secreted into the maternal circulation, where it is found complex-bound to pregnancy-associated plasma protein-A (PAPP-A) and other proteins. We examined the potential of proMBP as a maternal serum marker for fetal Down syndrome (DS) by determining its maternal serum concentration (MSpMBP) in 25 Down syndrome (DS) pregnancies and 152 control pregnancies in the first trimester, and in 105 DS pregnancies and 156 control pregnancies in the second trimester. The median (95 per cent confidence interval) MSpMBP MoM in DS pregnancies (n=15) was 0.66 (0.49-0.79) in gestational weeks 5-9; 1.06 (0.71-1.97) in weeks 10-12 (n=10) and 1.62 (1.18-1.98) in weeks 14-20 (n=105). Using parameterized receiver operator characteristics analysis for proMBP as a single marker for DS, detection rates (DRs) of 22 per cent and 38 per cent, for false-positive rates (FPRs) of 5 per cent, were found in weeks 5-9 (using MSpMBP</=cut-off) and weeks 14-20 (using MSpMBP>/=cut-off), respectively. When age and MSpMBP were used as markers in combination, a DR of 36.8 per cent for an FPR of 5.5 per cent was obtained in weeks 5-9 using a risk cut-off of 1:250. In weeks 14-20 the DR was 48.4 per cent for an FPR of 5.3 per cent using the same risk cut-off. This makes proMBP a marker comparable in diagnostic efficiency to human chorionic gonadotrophin (hCG), and exceeding that of alpha-fetoprotein (AFP) and unconjugated oestriol (uE3), in the second trimester.
Collapse
Affiliation(s)
- M Christiansen
- Department of Clinical Biochemistry, Statens Serum Institut, Copenhagen, Denmark.
| | | | | | | | | | | | | | | | | | | |
Collapse
|
17
|
Abstract
The article reviews screening for Down syndrome in the first trimester (8-13 gestational weeks) with maternal serum analytes. In the first trimester, 2 serum markers stand out: pregnancy-associated plasma protein-A, a large glycoprotein tetramer, and free beta-human chorionic gonadotropin (beta-hCG), 1 of the 2 subunits of the glycoprotein hormone hCG. Some data indicate that hCG itself may be as effective as free beta-hCG in the first trimester. Maternal serum levels of pregnancy-associated plasma protein-A are low and free beta-hCG are high (consensus multiple of the medians, 0.4 and 1.8, respectively) in Down syndrome pregnancy. The consensus estimate of screening performance by using pregnancy-associated plasma protein-A and free beta-hCG in combination with maternal age is 60% detection rate at a 5% false positive rate. This is similar to the screening performance of second trimester double markers, but not as good as the screening performance of second trimester triple or quad markers. For this reason, first trimester screening with serum markers alone cannot be recommended except in cases in which second trimester screening cannot be done.
Collapse
Affiliation(s)
- J A Canick
- Women and Infants Hospital and the Department of Pathology and Laboratory Medicine, Brown University School of Medicine, Providence, RI 02905, USA.
| | | |
Collapse
|
18
|
Abstract
Meta-analysis was used to calculate maternal serum marker distribution parameters for Down syndrome risk estimation in the first trimester. Data from 44 series were combined: relating to pregnancy associated plasma protein (PAPP)-A in 18, free beta human chorionic gonadotrophin (hCG) in 17, alpha-fetoprotein (AFP) in 26 and unconjugated oestriol (uE3) in 9. All levels were expressed in multiples of the normal median (MOM) for gestational age. Individual PAPP-A levels were available for 439 first and second-trimester Down syndrome pregnancies. The median MOM value increased with gestation: 0.35 at 6-8 weeks (31 cases), 0.40 at 9-11 weeks (197), 0.62 at 12-14 weeks (113) and 0.94 thereafter (98). A cubic regression equation was fitted so it could be estimated for each week of gestation. For the other markers the median value in Down syndrome was estimated from the weighted mean across all first-trimester series: 1.98 MOM for free beta-hCG in 579 cases; 0.79 MOM for AFP in 243 and 0.74 MOM for uE3 in 226. Variance-covariance matrices were calculated directly in unaffected pregnancies and from the difference between affected and unaffected pregnancies in Down syndrome. Based on these parameters we estimate that screening at 9-11 weeks with PAPP-A and free beta-hCG will yield a 64.6 per cent detection rate for a 5 per cent false-positive rate. Adding a third marker will increase detection to 66.6 per cent for AFP and 68.6 per cent for uE3; using all four markers it increases to 70.1 per cent. Routine ultrasound nuchal translucency measurement in addition to serum testing will increase the rates to 86.4 per cent, 87.2 per cent, 87.9 per cent and 88.3 per cent, respectively.
Collapse
Affiliation(s)
- H S Cuckle
- Centre for Reproduction, Growth and Development, School of Medicine, University of Leeds, UK.
| | | |
Collapse
|
19
|
Abstract
We have carried out a large retrospective study of alpha-fetoprotein (AFP), free-beta human chorionic gonadotrophin (hCG) and pregnancy-associated plasma protein (PAPP-A) in the first trimester of pregnancy. Unlike other studies all women had routine ultrasound dating, carried out during a nuchal translucency measurement project. A total of 13,477 serum samples were tested for AFP and 11,659 for free beta-hCG. A subset of 1564 samples from unaffected pregnancies were also tested for PAPP-A on a case-control basis. All three markers were also determined in 31 samples from pregnancies with Down syndrome. Equations were derived to express results in multiples of the median using both gestational age and crown rump length and to adjust for maternal weight. Statistical modelling with Gaussian distribution parameters obtained in the study were used to predict the detection rate for a 5 per cent false-positive rate. The predicted rates were: 73.7 per cent for all three markers; 69.1 per cent for PAPP-A and free beta-hCG; 47.4 per cent for PAPP-A and AFP; 57.6 per cent for free beta-hCG and AFP. As these rates are similar to those in the second trimester, health planners may now want to consider a change in policy from second-trimester to first-trimester screening with biochemical markers.
Collapse
Affiliation(s)
- G L Tsukerman
- Institute for Hereditary Diseases, Centre for Medical Genetic Services, Minsk, Belarus
| | | | | |
Collapse
|
20
|
de Graaf IM, Pajkrt E, Bilardo CM, Leschot NJ, Cuckle HS, van Lith JM. Early pregnancy screening for fetal aneuploidy with serum markers and nuchal translucency. Prenat Diagn 1999; 19:458-62. [PMID: 10360515 DOI: 10.1002/(sici)1097-0223(199905)19:5<458::aid-pd569>3.0.co;2-a] [Citation(s) in RCA: 65] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
We determined the aneuploidy detection rate achievable by early pregnancy screening with pregnancy associated plasma protein (PAPP)-A, free beta human chorionic gonadotrophin (hCG) and ultrasound nuchal translucency (NT) measurement. Women having prenatal diagnosis were scanned, and a blood sample was taken and stored. Stored samples were tested and a total of 37 were found to have Down syndrome, 8 to have Edwards syndrome and 255 were controls. Results were expressed in multiples of the gestation-specific median (MOM) value in the controls after regression and, for the serum markers, maternal weight adjustment. In Down syndrome the medians were for PAPP-A 0.63 MOM (95 per cent confidence interval (CI) 0.45-0.87); free beta-hCG 1.88 MOM (1.33-2.66); and NT 2.34 MOM (1.70-3.22). Using these parameters the expected detection rate for a 5 per cent false-positive rate for different marker combinations were: 55.3 per cent for PAPP-A and free beta-hCG; 68.4 per cent for NT alone; and 84.6 per cent for PAPP-A, free beta-hCG and NT. The median values for Edwards syndrome were: 0.17 MOM for PAPP-A; 0.18 MOM for free beta-hCG; and 2.64 MOM for NT. Early pregnancy screening with the combined measurement of maternal serum PAPP-A and free beta-hCG and fetal nuchal translucency could achieve a high Down syndrome detection rate.
Collapse
Affiliation(s)
- I M de Graaf
- Department of Obstetrics and Gynaecology, Academic Medical Centre, Amsterdam, The Netherlands.
| | | | | | | | | | | |
Collapse
|
21
|
Massé J, Forest JC, Moutquin JM, Degrandpre P. Effect of repeated freeze-thaw cycles on maternal serum biological markers for the detection of fetal trisomies. Clin Biochem 1997; 30:527-30. [PMID: 9399020 DOI: 10.1016/s0009-9120(97)00095-7] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/05/2023]
Abstract
OBJECTIVES We studied the stability of maternal blood markers for screening for Down syndrome (alpha-fetoprotein, unconjugated estriol, intact human chorionic gonadotropin (hCG) and free beta-human chorionic gonadotropin) upon repeated freeze-thaw cycles. DESIGN AND METHODS Forty-three samples collected from second trimester normal pregnancies were submitted to five freeze-thaw cycles. After each cycle, the markers were measured using kits and instruments from Wallac Canada (AutoDelfia). Results were compared by repeated measures analysis of variance and by analysis of linear trend (after mathematical transformation of the results in order to decrease between-sample variation) as a function of the number of freeze-thaw cycles. RESULTS No significant differences were observed by ANOVA (p > 0.1) for any marker. Intact hCG showed a statistically significant linear downward trend (slope = -0.0063, p = 0.009) while free beta-hCG increased (slope = 0.011, p = 0.004). After five freeze-thaw cycles, a mean decrease of 3.2% is predicted for intact hCG while free beta-hCG would increase by 5.5% on average. CONCLUSION We conclude that the studied markers do not show clinically significant changes under the evaluated conditions. The observed changes of intact hCG and free beta-hCG would have a limited impact on the screening performance.
Collapse
Affiliation(s)
- J Massé
- Department of Biochemistry, Faculty of Medicine, Laval University and Research Centre, C.H.U.Q., Quebec City, Canada.
| | | | | | | |
Collapse
|