The Clinician-reported Genetic testing Utility InDEx (C-GUIDE): Preliminary evidence of validity and reliability

Opportunistic genomic screening has clinical utility: An interventional cohort study

PURPOSE

Practice is shifting toward genome-first approaches, such as opportunistic screening for secondary findings (SFs). Analysis of SFs could be extended beyond medically actionable results to include non-medically actionable monogenic disease risks, carrier status, pharmacogenomic variants, and risk variants for common complex disease. However, evidence on the clinical utility of returning these results is lacking. We assessed the outcomes of opportunistic screening for a broad spectrum of SFs by evaluating the yield, impact on clinical management, and consistency between SFs and participants' clinical features and family history.

METHODS

Adult cancer patients had exome sequencing with the option to learn multiple categories of SFs. Outcomes data were collected through chart review and participant-reported measures up to one year after return of results.

RESULTS

All participants (n = 139, 85.6% female, average 54.6 years old) who elected to learn SFs had ≥1 variant reported (100% [139/139]). The yield of reportable findings was highest for pharmacogenomic variants (97.8% [135/138] of participants), followed by common disease risk variants (89.4% [118/132]), carrier status (89.3% [117/131]), and variants related to Mendelian (27.2% [34/125]), medically actionable (15.2% [21/138]), and early-onset neurodegenerative (2.6% [3/117]) disease risks. SFs from the American College of Medical Genetics and Genomics list (v3.2, noncancer genes) were reported in 1.4% (2/138) of participants. SFs across all categories demonstrated clinical utility by prompting management changes in 28.1% (39/139) of participants. Moreover, a considerable proportion of participants had suggestive clinical features (49.0% (24/49)]) or family history (21.8% (27/124)) potentially related to their SFs.

CONCLUSION

Our findings indicate there are potential benefits from opportunistic screening for a broad range of SFs.

The Clinician-reported Genetic Testing Utility Index (C-GUIDE) for Prenatal Care: Initial evidence of content and construct validity

PURPOSE

To develop and assess the face and construct validity of the Clinician-reported Genetic Testing Utility Index (C-GUIDE) for genetic testing in prenatal care.

METHODS

After a literature review and consultation with clinical experts, a preliminary draft of C-GUIDE Prenatal was developed. Its face and content validity were then assessed by 19 prenatal genetics' providers using interviews and surveys. Feedback informed further revisions. To test construct validity, 4 geneticist raters completed C-GUIDE on a retrospective sample of cases that received prenatal genetic testing and completed a concurrent global assessment of utility of these cases using an anchor item. A generalized estimating equations model was used to adjust for rater correlation and measure the association between C-GUIDE scores, global item scores, and potential clinical variables.

RESULTS

To develop C-GUIDE Prenatal, 7 items were removed, 10 items were modified, and 4 items were added. For 101 cases rated for validation, on average, a 1-point increase in the global item score was associated with an increase of 1.1 in the C-GUIDE score (P = .04). Compared with uninformative results, informative positive and informative negative results were associated with a mean increase of 10.7 (SE = 1.05) (P < .001) and 5.6 (SE = 1.85) (P < .001), respectively. As indications for testing, known/familial variants were associated with a mean increase in the C-GUIDE score of 4.7 (SE = 2.21) (P < .001) compared with ultrasound findings. C-GUIDE scores increased by a mean of 3.0 (SE = 0.23) among cases for whom pregnancies were ongoing compared with those for whom they were not (P < .01).

CONCLUSION

The significant positive associations between C-GUIDE total and the global item score and between C-GUIDE total, result type, indication for testing, and pregnancy status in the expected directions provide evidence of construct validity.

NeoGx: Machine-Recommended Rapid Genome Sequencing for Neonates

Background

Genetic disease is common in the Level IV Neonatal Intensive Care Unit (NICU), but neonatology providers are not always able to identify the need for genetic evaluation. We trained a machine learning (ML) algorithm to predict the need for genetic testing within the first 18 months of life using health record phenotypes.

Methods

For a decade of NICU patients, we extracted Human Phenotype Ontology (HPO) terms from clinical text with Natural Language Processing tools. Considering multiple feature sets, classifier architectures, and hyperparameters, we selected a classifier and made predictions on a validation cohort of 2,241 Level IV NICU admits born 2020-2021.

Results

Our classifier had ROC AUC of 0.87 and PR AUC of 0.73 when making predictions during the first week in the Level IV NICU. We simulated testing policies under which subjects begin testing at the time of first ML prediction, estimating diagnostic odyssey length both with and without the additional benefit of pursuing rGS at this time. Just by using ML to accelerate initial genetic testing (without changing the tests ordered), the median time to first genetic test dropped from 10 days to 1 day, and the number of diagnostic odysseys resolved within 14 days of NICU admission increased by a factor of 1.8. By additionally requiring rGS at the time of positive ML prediction, the number of diagnostic odysseys resolved within 14 days was 3.8 times higher than the baseline.

Conclusions

ML predictions of genetic testing need, together with the application of the right rapid testing modality, can help providers accelerate genetics evaluation and bring about earlier and better outcomes for patients.

Measuring health-related quality of life in children with suspected genetic conditions: validation of the PedsQL proxy-report versions

PURPOSE

Measuring health-related quality of life (HRQoL) of children with suspected genetic conditions is important for understanding the effect of interventions such as genomic sequencing (GS). The Pediatric Quality of Life Inventory (PedsQL) is a widely used generic measure of HRQoL in pediatric patients, but its psychometric properties have not yet been evaluated in children undergoing diagnostic GS.

METHODS

In this cross-sectional study, we surveyed caregivers at the time of their child's enrollment into GS research studies as part of the Clinical Sequencing Evidence Generating Research (CSER) consortium. To evaluate structural validity of the PedsQL 4.0 Generic Core Scales and PedsQL Infant Scales parent proxy-report versions, we performed a confirmatory factor analysis of the hypothesized factor structure. To evaluate convergent validity, we examined correlations between caregivers' reports of their child's health, assessed using the EQ VAS, and PedsQL scores by child age. We conducted linear regression analyses to examine whether age moderated the association between caregiver-reported child health and PedsQL scores. We assessed reliability using Cronbach's alpha.

RESULTS

We analyzed data for 766 patients across all PedsQL age group versions (1-12 months through 13-18 years). Model fit failed to meet criteria for good fit, even after modification. Neither age group (categorical) nor age (continuous) significantly moderated associations between PedsQL scores and caregiver-reported child health. Cronbach's alphas indicated satisfactory internal consistency for most PedsQL scales.

CONCLUSION

The PedsQL Generic Core Scales and Infant Scales may be appropriate to measure HRQoL in pediatric patients with suspected genetic conditions across a wide age range. While we found evidence of acceptable internal consistency and preliminary convergent validity in this sample, there were some potential problems with structural validity and reliability that require further attention.

Determining priority indicators of utility for genomic testing in rare disease: A Delphi study

PURPOSE

Determining the value of genomic tests in rare disease necessitates a broader conceptualization of genomic utility beyond diagnostic yield. Despite widespread discussion, consensus toward which aspects of value to consider is lacking. This study aimed to use expert opinion to identify and refine priority indicators of utility in rare disease genomic testing.

METHODS

We used 2 survey rounds following Delphi methodology to obtain consensus on indicators of utility among experts involved in policy, clinical, research, and consumer advocacy leadership in Australia. We analyzed quantitative and qualitative data to identify, define, and determine priority indicators.

RESULTS

Twenty-five experts completed round 1 and 18 completed both rounds. Twenty indicators reached consensus as a priority in value assessment, including those relating to prognostic information, timeliness of results, practical and health care outcomes, clinical accreditation, and diagnostic yield. Whereas indicators pertaining to discovery research, disutility, and factors secondary to primary reason for testing were considered less of a priority and were removed.

CONCLUSION

This study obtained expert consensus on different utility indicators that are considered a priority in determining the value of genomic testing in rare disease in Australia. Indicators may inform a standardized approach to evidence generation and assessment to guide future research, decision making, and implementation efforts.

National Rapid Genome Sequencing in Neonatal Intensive Care

Importance

National implementation of rapid trio genome sequencing (rtGS) in a clinical acute setting is essential to ensure advanced and equitable care for ill neonates.

Objective

To evaluate the feasibility, diagnostic efficacy, and clinical utility of rtGS in neonatal intensive care units (NICUs) throughout Israel.

Design, Setting, and Participants

This prospective, public health care-based, multicenter cohort study was conducted from October 2021 to December 2022 with the Community Genetics Department of the Israeli Ministry of Health and all Israeli medical genetics institutes (n = 18) and NICUs (n = 25). Critically ill neonates suspected of having a genetic etiology were offered rtGS. All sequencing, analysis, and interpretation of data were performed in a central genomics center at Tel-Aviv Sourasky Medical Center. Rapid results were expected within 10 days. A secondary analysis report, issued within 60 days, focused mainly on cases with negative rapid results and actionable secondary findings. Pathogenic, likely pathogenic, and highly suspected variants of unknown significance (VUS) were reported.

Main Outcomes and Measures

Diagnostic rate, including highly suspected disease-causing VUS, and turnaround time for rapid results. Clinical utility was assessed via questionnaires circulated to treating neonatologists.

Results

A total of 130 neonates across Israel (70 [54%] male; 60 [46%] female) met inclusion criteria and were recruited. Mean (SD) age at enrollment was 12 (13) days. Mean (SD) turnaround time for rapid report was 7 (3) days. Diagnostic efficacy was 50% (65 of 130) for disease-causing variants, 11% (14 of 130) for VUS suspected to be causative, and 1 novel gene candidate (1%). Disease-causing variants included 12 chromosomal and 52 monogenic disorders as well as 1 neonate with uniparental disomy. Overall, the response rate for clinical utility questionnaires was 82% (107 of 130). Among respondents, genomic testing led to a change in medical management for 24 neonates (22%). Results led to immediate precision medicine for 6 of 65 diagnosed infants (9%), an additional 2 (3%) received palliative care, and 2 (3%) were transferred to nursing homes.

Conclusions and Relevance

In this national cohort study, rtGS in critically ill neonates was feasible and diagnostically beneficial in a public health care setting. This study is a prerequisite for implementation of rtGS for ill neonates into routine care and may aid in design of similar studies in other public health care systems.

Determining the utility of diagnostic genomics: a conceptual framework

BACKGROUND

Diagnostic efficacy is now well established for diagnostic genomic testing in rare disease. Assessment of overall utility is emerging as a key next step, however ambiguity in the conceptualisation and measurement of utility has impeded its assessment in a comprehensive manner. We propose a conceptual framework to approach determining the broader utility of diagnostic genomics encompassing patients, families, clinicians, health services and health systems to assist future evidence generation and funding decisions. BODY: Building upon previous work, our framework posits that utility of diagnostic genomics consists of three dimensions: the domain or type and extent of utility (what), the relationship and perspective of utility (who), and the time horizon of utility (when). Across the description, assessment, and summation of these three proposed dimensions of utility, one could potentially triangulate a singular point of utility axes of type, relationship, and time. Collectively, the multiple different points of individual utility might be inferred to relate to a concept of aggregate utility.

CONCLUSION

This ontological framework requires retrospective and prospective application to enable refinement and validation. Moving forward our framework, and others which have preceded it, promote a better characterisation and description of genomic utility to inform decision-making and optimise the benefits of genomic diagnostic testing.

Assessing the Performance of the Clinician-reported Genetic Testing Utility InDEx (C-GUIDE): Further Evidence of Inter-rater Reliability

PURPOSE

Advanced genomic and genetic testing technologies are quickly diffusing into clinical practice, but standardized approaches to assessing their clinical utility are limited. Previous work developed and generated preliminary evidence of validity for a novel outcome measure, the Clinician-reported Genetic testing Utility InDEx (C-GUIDE). C-GUIDE is a 17-item measure that captures the utility of genetic testing from the providers' perspective. Preliminary evidence of its inter-rater reliability was obtained through a clinical vignette study. The purpose of this study was to further assess its inter-rater reliability using actual clinical cases.

METHODS

One genetic counselor and one medical geneticist independently completed C-GUIDE Version 1.1 after genetic test results were disclosed to a shared set of 42 patients. Raters also completed a case description questionnaire, including information about the patient's age, indication for testing, and type of test performed. Inter-rater reliability was assessed by comparing the raters' C-GUIDE scores using ANOVA to generate intra-class correlation coefficients (ICCs), absolute agreement, and mixed repeated measures ANOVA.

FINDINGS

Of the 42 patients studied, the most common indications for testing were hearing loss (n = 18) and craniosynostosis (n = 11), and the most common tests ordered were gene panels (n = 20) and microarrays (n = 10). Test results were diagnostic or partially diagnostic for 11 patients, potentially diagnostic for 14 patients, or nondiagnostic for 17 patients. The overall ICC was 0.95 (95% CI, 0.89-0.97) and absolute agreement was acceptable (>70%) for 15 individual items. Inter-rater agreement was excellent (ICC > 0.90) for 8 items, good (ICC = 0.75-0.89) for 3 items, moderate (ICC = 0.50-0.74) for 4 items and poor (ICC < 0.50) for 2 items. Absolute agreement was unacceptable (<70%), and rater agreement was fair (ICC = 0.40-0.59) for 2 items. For the global rating, the ICC was 0.62 (95% CI, 0.39-0.77), and the absolute agreement was 61.9%.

IMPLICATIONS

Rater instructions for item completion have been modified to improve consistency of item interpretation. Although further assessments of reliability are warranted after modifications, these findings provide additional tentative evidence of C-GUIDE's inter-rater reliability and suggest that it may be useful as a strategy for measuring the value of genetic testing, as perceived by genetics providers.

The PrU: Development and validation of a measure to assess personal utility of genomic results

PURPOSE

People report experiencing value from learning genomic results even in the absence of clinically actionable information. Such personal utility has emerged as a key concept in genomic medicine. The lack of a validated patient-reported outcome measure of personal utility has impeded the ability to assess this concept among those receiving genomic results and evaluate the patient-perceived value of genomics. We aimed to construct and psychometrically evaluate a scale to measure personal utility of genomic results-the Personal Utility (PrU) scale.

METHODS

We used an evidence-based, operational definition of personal utility, with data from a systematic literature review and Delphi survey to build a novel scale. After piloting with 24 adults, the PrU was administered to healthy adults in a Clinical Sequencing Evidence-Generating Research Consortium study after receiving results. We investigated the responses using exploratory factor analysis.

RESULTS

The exploratory factor analysis (N = 841 participants) resulted in a 3-factor solution, accounting for 74% of the variance in items: (1) self-knowledge (α = 0.92), (2) reproductive planning (α = 0.89), and (3) practical benefits (α = 0.91).

CONCLUSION

Our findings support the use of the 3-factor PrU to assess personal utility of genomic results. Validation of the PrU in other samples will be important for more wide-spread application.

Combining cytogenetic and genomic technologies for deciphering challenging complex chromosomal rearrangements

Complex chromosomal rearrangements (CCRs), a class of structural variants (SVs) involving more than two chromosome breaks, were classically thought to be extremely rare. As advanced technologies become more available, it has become apparent that CCRs are more common than formerly thought, and are a substantial cause of genetic disorders. We attempted a novel approach for solving the mechanism of challenging CCRs, which involve repetitive sequences, by precisely identifying sequence-level changes and their order. Chromosomal microarray (CMA) and FISH analyses were used for interpretation of SVs detected by whole exome sequencing (WES). Breakpoint junctions were analyzed by Nanopore sequencing, a novel long-read whole genome sequencing tool. A large deletion identified by WES, encompassing the FOXF1 enhancer, was the cause of alveolar capillary dysplasia and respiratory insufficiency, resulting in perinatal death. CMA analysis of the newborn's mother revealed two duplications encompassing the deleted region in the proband, raising our hypothesis that the deletion resulted from the mother's CCR. Breakpoint junctions of complex SVs were determined at the nucleotide level using Nanopore long-read sequencing. According to sequencing results of breakpoint junctions, the CCR in the newborn was considered the consequence of at least one double-strand break during meiosis, and reassembly of DNA fragments by intra-chromosomal homologous recombination. Our comprehensive approach, combining cytogenetics and long-read sequencing, enabled delineation of the exact breakpoints in a challenging CCR, and proposal of a mechanism in which it arises. We suggest applying our integrative approach combining technologies for deciphering future challenging CCRs, enabling risk assessment in families.