Connectome caricatures: removing large-amplitude co-activation patterns in resting-state fMRI emphasizes individual differences

High-amplitude co-activation patterns are sparsely present during resting-state fMRI but drive functional connectivity 1-5 . Further, they resemble task activation patterns and are well-studied 3,5-10 . However, little research has characterized the remaining majority of the resting-state signal. In this work, we introduced caricaturing-a method to project resting-state data to a subspace orthogonal to a manifold of co-activation patterns estimated from the task fMRI data. Projecting to this subspace removes linear combinations of these co-activation patterns from the resting-state data to create Caricatured connectomes. We used rich task data from the Human Connectome Project (HCP) 11 and the UCLA Consortium for Neuropsychiatric Phenomics 12 to construct a manifold of task co-activation patterns. Caricatured connectomes were created by projecting resting-state data from the HCP and the Yale Test-Retest 13 datasets away from this manifold. Like caricatures, these connectomes emphasized individual differences by reducing between-individual similarity and increasing individual identification 14 . They also improved predictive modeling of brain-phenotype associations. As caricaturing removes group-relevant task variance, it is an initial attempt to remove task-like co-activations from rest. Therefore, our results suggest that there is a useful signal beyond the dominating co-activations that drive resting-state functional connectivity, which may better characterize the brain's intrinsic functional architecture.

Data leakage inflates prediction performance in connectome-based machine learning models

Predictive modeling is a central technique in neuroimaging to identify brain-behavior relationships and test their generalizability to unseen data. However, data leakage undermines the validity of predictive models by breaching the separation between training and test data. Leakage is always an incorrect practice but still pervasive in machine learning. Understanding its effects on neuroimaging predictive models can inform how leakage affects existing literature. Here, we investigate the effects of five forms of leakage-involving feature selection, covariate correction, and dependence between subjects-on functional and structural connectome-based machine learning models across four datasets and three phenotypes. Leakage via feature selection and repeated subjects drastically inflates prediction performance, whereas other forms of leakage have minor effects. Furthermore, small datasets exacerbate the effects of leakage. Overall, our results illustrate the variable effects of leakage and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Brain-phenotype predictions can survive across diverse real-world data

Recent work suggests that machine learning models predicting psychiatric treatment outcomes based on clinical data may fail when applied to unharmonized samples. Neuroimaging predictive models offer the opportunity to incorporate neurobiological information, which may be more robust to dataset shifts. Yet, among the minority of neuroimaging studies that undertake any form of external validation, there is a notable lack of attention to generalization across dataset-specific idiosyncrasies. Research settings, by design, remove the between-site variations that real-world and, eventually, clinical applications demand. Here, we rigorously test the ability of a range of predictive models to generalize across three diverse, unharmonized samples: the Philadelphia Neurodevelopmental Cohort (n=1291), the Healthy Brain Network (n=1110), and the Human Connectome Project in Development (n=428). These datasets have high inter-dataset heterogeneity, encompassing substantial variations in age distribution, sex, racial and ethnic minority representation, recruitment geography, clinical symptom burdens, fMRI tasks, sequences, and behavioral measures. We demonstrate that reproducible and generalizable brain-behavior associations can be realized across diverse dataset features with sample sizes in the hundreds. Results indicate the potential of functional connectivity-based predictive models to be robust despite substantial inter-dataset variability. Notably, for the HCPD and HBN datasets, the best predictions were not from training and testing in the same dataset (i.e., cross-validation) but across datasets. This result suggests that training on diverse data may improve prediction in specific cases. Overall, this work provides a critical foundation for future work evaluating the generalizability of neuroimaging predictive models in real-world scenarios and clinical settings.

Test-Retest Reliability of Functional Connectivity in Adolescents With Depression

BACKGROUND

The test-retest reliability of functional magnetic resonance imaging is critical to identifying reproducible biomarkers for psychiatric illness. Recent work has shown how reliability limits the observable effect size of brain-behavior associations, hindering detection of these effects. However, while a fast-growing literature has explored both univariate and multivariate reliability in healthy individuals, relatively few studies have explored reliability in populations with psychiatric illnesses or how this interacts with age.

METHODS

Here, we investigated functional connectivity reliability over the course of 1 year in a longitudinal cohort of 88 adolescents (age at baseline = 15.63 ± 1.29 years; 64 female) with major depressive disorder (MDD) and without MDD (healthy volunteers [HVs]). We compared a univariate metric, intraclass correlation coefficient, and 2 multivariate metrics, fingerprinting and discriminability.

RESULTS

Adolescents with MDD had marginally higher mean intraclass correlation coefficient (μMDD = 0.34, 95% CI, 0.12-0.54; μHV = 0.27, 95% CI, 0.05-0.52), but both groups had poor average intraclass correlation coefficients (<0.4). Fingerprinting index was greater than chance and did not differ between groups (fingerprinting indexMDD = 0.75; fingerprinting indexHV = 0.91; Poisson tests p < .001). Discriminability indicated high multivariate reliability in both groups (discriminabilityMDD = 0.80; discriminabilityHV = 0.82; permutation tests p < .01). Neither univariate nor multivariate reliability was associated with symptom severity or edge-level effect size of group differences.

CONCLUSIONS

Overall, we found little evidence for a relationship between depression and reliability of functional connectivity during adolescence. These findings suggest that biomarker identification in depression is not limited due to reliability compared with healthy samples and support the shift toward multivariate analysis for improved power and reliability.

The effects of data leakage on connectome-based machine learning models

Predictive modeling has now become a central technique in neuroimaging to identify complex brain-behavior relationships and test their generalizability to unseen data. However, data leakage, which unintentionally breaches the separation between data used to train and test the model, undermines the validity of predictive models. Previous literature suggests that leakage is generally pervasive in machine learning, but few studies have empirically evaluated the effects of leakage in neuroimaging data. Although leakage is always an incorrect practice, understanding the effects of leakage on neuroimaging predictive models provides insight into the extent to which leakage may affect the literature. Here, we investigated the effects of leakage on machine learning models in two common neuroimaging modalities, functional and structural connectomes. Using over 400 different pipelines spanning four large datasets and three phenotypes, we evaluated five forms of leakage fitting into three broad categories: feature selection, covariate correction, and lack of independence between subjects. As expected, leakage via feature selection and repeated subjects drastically inflated prediction performance. Notably, other forms of leakage had only minor effects (e.g., leaky site correction) or even decreased prediction performance (e.g., leaky covariate regression). In some cases, leakage affected not only prediction performance, but also model coefficients, and thus neurobiological interpretations. Finally, we found that predictive models using small datasets were more sensitive to leakage. Overall, our results illustrate the variable effects of leakage on prediction pipelines and underscore the importance of avoiding data leakage to improve the validity and reproducibility of predictive modeling.

Power and reproducibility in the external validation of brain-phenotype predictions

Identifying reproducible and generalizable brain-phenotype associations is a central goal of neuroimaging. Consistent with this goal, prediction frameworks evaluate brain-phenotype models in unseen data. Most prediction studies train and evaluate a model in the same dataset. However, external validation, or the evaluation of a model in an external dataset, provides a better assessment of robustness and generalizability. Despite the promise of external validation and calls for its usage, the statistical power of such studies has yet to be investigated. In this work, we ran over 60 million simulations across several datasets, phenotypes, and sample sizes to better understand how the sizes of the training and external datasets affect statistical power. We found that prior external validation studies used sample sizes prone to low power, which may lead to false negatives and effect size inflation. Furthermore, increases in the external sample size led to increased simulated power directly following theoretical power curves, whereas changes in the training dataset size offset the simulated power curves. Finally, we compared the performance of a model within a dataset to the external performance. The within-dataset performance was typically within r=0.2 of the cross-dataset performance, which could help decide how to power future external validation studies. Overall, our results illustrate the importance of considering the sample sizes of both the training and external datasets when performing external validation.

Altered Brain Dynamics Across Bipolar Disorder and Schizophrenia During Rest and Task Switching Revealed by Overlapping Brain States

BACKGROUND

Individuals with bipolar disorder (BD) and schizophrenia (SCZ) show aberrant brain dynamics (i.e., altered recruitment or traversal through different brain states over time). Existing investigations of brain dynamics typically assume that one dominant brain state characterizes each time point. However, as multiple brain states likely are engaged at any given moment, this approach can obscure alterations in less prominent but critical brain states. Here, we examined brain dynamics in BD and SCZ by implementing a novel framework that simultaneously assessed the engagement of multiple brain states.

METHODS

Four recurring brain states were identified by applying nonlinear manifold learning and k-means clustering to the Human Connectome Project task-based functional magnetic resonance imaging data. We then assessed moment-to-moment state engagement in 2 independent samples of healthy control participants and patients with BD or SCZ using resting-state (N = 336) or task-based (N = 217) functional magnetic resonance imaging data. Relative state engagement and state engagement variability were extracted and compared across groups using multivariate analysis of covariance, controlling for site, medication, age, and sex.

RESULTS

Our framework identified dynamic alterations in BD and SCZ, while a state discretization approach revealed no significant group differences. Participants with BD or SCZ showed reduced state engagement variability, but not relative state engagement, across multiple brain states during resting-state and task-based functional magnetic resonance imaging. We found decreased state engagement variability in older participants and preliminary evidence suggesting an association with avolition.

CONCLUSIONS

Assessing multiple brain states simultaneously can reflect the complexity of aberrant brain dynamics in BD and SCZ, providing a more comprehensive understanding of the neural mechanisms underpinning these conditions.

Network controllability of structural connectomes in the neonatal brain

White matter connectivity supports diverse cognitive demands by efficiently constraining dynamic brain activity. This efficiency can be inferred from network controllability, which represents the ease with which the brain moves between distinct mental states based on white matter connectivity. However, it remains unclear how brain networks support diverse functions at birth, a time of rapid changes in connectivity. Here, we investigate the development of network controllability during the perinatal period and the effect of preterm birth in 521 neonates. We provide evidence that elements of controllability are exhibited in the infant's brain as early as the third trimester and develop rapidly across the perinatal period. Preterm birth disrupts the development of brain networks and altered the energy required to drive state transitions at different levels. In addition, controllability at birth is associated with cognitive ability at 18 months. Our results suggest network controllability develops rapidly during the perinatal period to support cognitive demands but could be altered by environmental impacts like preterm birth.

Cross Atlas Remapping via Optimal Transport (CAROT): Creating connectomes for different atlases when raw data is not available

Open-source, publicly available neuroimaging datasets - whether from large-scale data collection efforts or pooled from multiple smaller studies - offer unprecedented sample sizes and promote generalization efforts. Releasing data can democratize science, increase the replicability of findings, and lead to discoveries. Partly due to patient privacy, computational, and data storage concerns, researchers typically release preprocessed data with the voxelwise time series parcellated into a map of predefined regions, known as an atlas. However, releasing preprocessed data also limits the choices available to the end-user. This is especially true for connectomics, as connectomes created from different atlases are not directly comparable. Since there exist several atlases with no gold standards, it is unrealistic to have processed, open-source data available from all atlases. Together, these limitations directly inhibit the potential benefits of open-source neuroimaging data. To address these limitations, we introduce Cross Atlas Remapping via Optimal Transport (CAROT) to find a mapping between two atlases. This approach allows data processed from one atlas to be directly transformed into a connectome based on another atlas without the need for raw data access. To validate CAROT, we compare reconstructed connectomes against their original counterparts (i.e., connectomes generated directly from an atlas), demonstrate the utility of transformed connectomes in downstream analyses, and show how a connectome-based predictive model can generalize to publicly available data that was processed with different atlases. Overall, CAROT can reconstruct connectomes from an extensive set of atlases - without needing the raw data - allowing already processed connectomes to be easily reused in a wide range of analyses while eliminating redundant processing efforts. We share this tool as both source code and as a stand-alone web application (http://carotproject.com/).

Connectome-based machine learning models are vulnerable to subtle data manipulations

Neuroimaging-based predictive models continue to improve in performance, yet a widely overlooked aspect of these models is "trustworthiness," or robustness to data manipulations. High trustworthiness is imperative for researchers to have confidence in their findings and interpretations. In this work, we used functional connectomes to explore how minor data manipulations influence machine learning predictions. These manipulations included a method to falsely enhance prediction performance and adversarial noise attacks designed to degrade performance. Although these data manipulations drastically changed model performance, the original and manipulated data were extremely similar (r = 0.99) and did not affect other downstream analysis. Essentially, connectome data could be inconspicuously modified to achieve any desired prediction performance. Overall, our enhancement attacks and evaluation of existing adversarial noise attacks in connectome-based models highlight the need for counter-measures that improve the trustworthiness to preserve the integrity of academic research and any potential translational applications.

A functional connectome signature of blood pressure in >30 000 participants from the UK biobank

AIMS

Elevated blood pressure (BP) is a prevalent modifiable risk factor for cardiovascular diseases and contributes to cognitive decline in late life. Despite the fact that functional changes may precede irreversible structural damage and emerge in an ongoing manner, studies have been predominantly informed by brain structure and group-level inferences. Here, we aim to delineate neurobiological correlates of BP at an individual level using machine learning and functional connectivity.

METHODS AND RESULTS

Based on whole-brain functional connectivity from the UK Biobank, we built a machine learning model to identify neural representations for individuals' past (∼8.9 years before scanning, N = 35 882), current (N = 31 367), and future (∼2.4 years follow-up, N = 3 138) BP levels within a repeated cross-validation framework. We examined the impact of multiple potential covariates, as well as assessed these models' generalizability across various contexts.The predictive models achieved significant correlations between predicted and actual systolic/diastolic BP and pulse pressure while controlling for multiple confounders. Predictions for participants not on antihypertensive medication were more accurate than for currently medicated patients. Moreover, the models demonstrated robust generalizability across contexts in terms of ethnicities, imaging centres, medication status, participant visits, gender, age, and body mass index. The identified connectivity patterns primarily involved the cerebellum, prefrontal, anterior insula, anterior cingulate cortex, supramarginal gyrus, and precuneus, which are key regions of the central autonomic network, and involved in cognition processing and susceptible to neurodegeneration in Alzheimer's disease. Results also showed more involvement of default mode and frontoparietal networks in predicting future BP levels and in medicated participants.

CONCLUSION

This study, based on the largest neuroimaging sample currently available and using machine learning, identifies brain signatures underlying BP, providing evidence for meaningful BP-associated neural representations in connectivity profiles.

Associations of physical frailty with health outcomes and brain structure in 483 033 middle-aged and older adults: a population-based study from the UK Biobank

BACKGROUND

Physical frailty is a state of increased vulnerability to stressors and is associated with serious health issues. However, how frailty affects and is affected by numerous other factors, including mental health and brain structure, remains underexplored. We aimed to investigate the mutual effects of frailty and health using large, multidimensional data.

METHODS

For this population-based study, we used data from the UK Biobank to examine the pattern and direction of association between physical frailty and 325 health-related measures across multiple domains, using linear mixed-effect models and adjusting for numerous confounders. Participants were included if complete data were available for all five indicators of frailty, all covariates, and at least one health measure. We further examined the association between frailty and brain structure and the role of this association in mediating the relationship between frailty and health outcomes.

FINDINGS

483 033 participants aged 38-73 years were included in the study at baseline (between Dec 19, 2006, and Oct 1, 2010); at a median follow-up of 9 years (IQR 8-10), behavioural data were available for 46 501 participants and neuroimaging data for 40 210 participants. The severity of physical frailty was significantly associated with decreased cognitive performance (Cohen's d=0·025-0·162), increased early-life risks (d=0·026-0·111), unhealthy lifestyle (d=0·013-0·394), poor physical fitness (d=0·007-0·668), increased symptoms of poor mental health (d=0·032-0·607), severe environmental pollution (d=0·013-0·064), and adverse biochemical markers (d=0·025-0·198). Some associations were bidirectional, with the strongest effects on mental health measures. The severity of frailty correlated with increased total white matter hyperintensity and lower grey matter volume, particularly in subcortical regions (d=0·027-0·082), which significantly mediated the association between frailty and health-related outcomes, although the mediated effects were small.

INTERPRETATION

Physical frailty is associated with diverse unfavourable health-related outcomes, which can be mediated by differences in brain structure. Our findings offer a framework for guiding preventative strategies targeting both frailty and psychiatric disorders.

FUNDING

National Institute of Mental Health, National Science Foundation.

Machine Learning and Prediction in Fetal, Infant, and Toddler Neuroimaging: A Review and Primer

Predictive models in neuroimaging are increasingly designed with the intent to improve risk stratification and support interventional efforts in psychiatry. Many of these models have been developed in samples of children school-aged or older. Nevertheless, despite growing evidence that altered brain maturation during the fetal, infant, and toddler (FIT) period modulates risk for poor mental health outcomes in childhood, these models are rarely implemented in FIT samples. Applications of predictive modeling in children of these ages provide an opportunity to develop powerful tools for improved characterization of the neural mechanisms underlying development. To facilitate the broader use of predictive models in FIT neuroimaging, we present a brief primer and systematic review on the methods used in current predictive modeling FIT studies. Reflecting on current practices in more than 100 studies conducted over the past decade, we provide an overview of topics, modalities, and methods commonly used in the field and under-researched areas. We then outline ethical and future considerations for neuroimaging researchers interested in predicting health outcomes in early life, including researchers who may be relatively new to either advanced machine learning methods or using FIT data. Altogether, the last decade of FIT research in machine learning has provided a foundation for accelerating the prediction of early-life trajectories across the full spectrum of illness and health.

Functional brain networks reflect spatial and temporal autocorrelation

High-throughput experimental methods in neuroscience have led to an explosion of techniques for measuring complex interactions and multi-dimensional patterns. However, whether sophisticated measures of emergent phenomena can be traced back to simpler, low-dimensional statistics is largely unknown. To explore this question, we examined resting-state functional magnetic resonance imaging (rs-fMRI) data using complex topology measures from network neuroscience. Here we show that spatial and temporal autocorrelation are reliable statistics that explain numerous measures of network topology. Surrogate time series with subject-matched spatial and temporal autocorrelation capture nearly all reliable individual and regional variation in these topology measures. Network topology changes during aging are driven by spatial autocorrelation, and multiple serotonergic drugs causally induce the same topographic change in temporal autocorrelation. This reductionistic interpretation of widely used complexity measures may help link them to neurobiology.

Methods for think-aloud interviews in health-related resource-use research: the PECUNIA RUM instrument

BACKGROUND

The think-aloud (TA) approach is a qualitative research method that allows for gaining insight into thoughts and cognitive processes. It can be used to incorporate a respondent's perspective when developing resource-use measurement (RUM) instruments. Currently, the application of TA methods in RUM research is limited, and so is the guidance on how to use them. Transparent publication of TA methods for RUM in health economics studies, which is the aim of this paper, can contribute to reducing the aforementioned gap.

METHODS

Methods for conducting TA interviews were iteratively developed by a multi-national working group of health economists and additional qualitative research expertise was sought. TA interviews were conducted in four countries to support this process. A ten-step process was outlined in three parts: Part A 'before the interview' (including translation, recruitment, training), Part B 'during the interview' (including setting, opening, completing the instrument, open-ended questions, closing), and part C 'after the interview' (including transcription and data analysis, trustworthiness).

CONCLUSIONS

This manuscript describes the step-by-step approach for conducting multi-national TA interviews with potential respondents of the PECUNIA RUM instrument. It increases the methodological transparency in RUM development and reduces the knowledge gap of using qualitative research methods in health economics.

Global diversity in individualized cortical network topography

Individualized cortical network topography (ICNT) varies between people and exhibits great variability in the association networks in the human brain. However, these findings were mainly discovered in Western populations. It remains unclear whether and how ICNT is shaped by the non-Western populations. Here, we leveraged a multisession hierarchical Bayesian model to define individualized functional networks in White American and Han Chinese populations with data from both US and Chinese Human Connectome Projects. We found that both the size and spatial topography of individualized functional networks differed between White American and Han Chinese groups, especially in the heteromodal association cortex (including the ventral attention, control, language, dorsal attention, and default mode networks). Employing a support vector machine, we then demonstrated that ethnicity-related ICNT diversity can be used to identify an individual's ethnicity with high accuracy (74%, pperm < 0.0001), with heteromodal networks contributing most to the classification. This finding was further validated through mass-univariate analyses with generalized additive models. Moreover, we reveal that the spatial heterogeneity of ethnic diversity in ICNT correlated with fundamental properties of cortical organization, including evolutionary cortical expansion, brain myelination, and cerebral blood flow. Altogether, this case study highlights a need for more globally diverse and publicly available neuroimaging datasets.

Leveraging edge-centric networks complements existing network-level inference for functional connectomes

The human connectome is modular with distinct brain regions clustering together to form large-scale communities, or networks. This concept has recently been leveraged in novel inferencing procedures by averaging the edge-level statistics within networks to induce more powerful inferencing at the network level. However, these networks are constructed based on the similarity between pairs of nodes. Emerging work has described novel edge-centric networks, which instead use the similarity between pairs of edges to construct networks. In this work, we use these edge-centric networks in a network-level inferencing procedure and compare this novel method to traditional inferential procedures and the network-level procedure using node-centric networks. We use data from the Human Connectome Project, the Healthy Brain Network, and the Philadelphia Neurodevelopmental Cohort and use a resampling technique with various sample sizes (n=40, 80, 120) to probe the power and specificity of each method. Across datasets and sample sizes, using the edge-centric networks outperforms using node-centric networks for inference as well as edge-level FDR correction and NBS. Additionally, the edge-centric networks were found to be more consistent in clustering effect sizes of similar values as compared to node-centric networks, although node-centric networks often had a lower average within-network effect size variability. Together, these findings suggest that using edge-centric networks for network-level inference can procure relatively powerful results while remaining similarly accurate to the underlying edge-level effects across networks, complementing previous inferential methods.

Functional Connectome-Based Predictive Modeling in Autism

Autism is a heterogeneous neurodevelopmental condition, and functional magnetic resonance imaging-based studies have helped advance our understanding of its effects on brain network activity. We review how predictive modeling, using measures of functional connectivity and symptoms, has helped reveal key insights into this condition. We discuss how different prediction frameworks can further our understanding of the brain-based features that underlie complex autism symptomatology and consider how predictive models may be used in clinical settings. Throughout, we highlight aspects of study interpretation, such as data decay and sampling biases, that require consideration within the context of this condition. We close by suggesting exciting future directions for predictive modeling in autism.

Associations between grip strength, brain structure, and mental health in > 40,000 participants from the UK Biobank

BACKGROUND

Grip strength is a widely used and well-validated measure of overall health that is increasingly understood to index risk for psychiatric illness and neurodegeneration in older adults. However, existing work has not examined how grip strength relates to a comprehensive set of mental health outcomes, which can detect early signs of cognitive decline. Furthermore, whether brain structure mediates associations between grip strength and cognition remains unknown.

METHODS

Based on cross-sectional and longitudinal data from over 40,000 participants in the UK Biobank, this study investigated the behavioral and neural correlates of handgrip strength using a linear mixed effect model and mediation analysis.

RESULTS

In cross-sectional analysis, we found that greater grip strength was associated with better cognitive functioning, higher life satisfaction, greater subjective well-being, and reduced depression and anxiety symptoms while controlling for numerous demographic, anthropometric, and socioeconomic confounders. Further, grip strength of females showed stronger associations with most behavioral outcomes than males. In longitudinal analysis, baseline grip strength was related to cognitive performance at ~9 years follow-up, while the reverse effect was much weaker. Further, baseline neuroticism, health, and financial satisfaction were longitudinally associated with subsequent grip strength. The results revealed widespread associations between stronger grip strength and increased grey matter volume, especially in subcortical regions and temporal cortices. Moreover, grey matter volume of these regions also correlated with better mental health and considerably mediated their relationship with grip strength.

CONCLUSIONS

Overall, using the largest population-scale neuroimaging dataset currently available, our findings provide the most well-powered characterization of interplay between grip strength, mental health, and brain structure, which may facilitate the discovery of possible interventions to mitigate cognitive decline during aging.

Brain-phenotype models fail for individuals who defy sample stereotypes

Individual differences in brain functional organization track a range of traits, symptoms and behaviours1-12. So far, work modelling linear brain-phenotype relationships has assumed that a single such relationship generalizes across all individuals, but models do not work equally well in all participants13,14. A better understanding of in whom models fail and why is crucial to revealing robust, useful and unbiased brain-phenotype relationships. To this end, here we related brain activity to phenotype using predictive models-trained and tested on independent data to ensure generalizability15-and examined model failure. We applied this data-driven approach to a range of neurocognitive measures in a new, clinically and demographically heterogeneous dataset, with the results replicated in two independent, publicly available datasets16,17. Across all three datasets, we find that models reflect not unitary cognitive constructs, but rather neurocognitive scores intertwined with sociodemographic and clinical covariates; that is, models reflect stereotypical profiles, and fail when applied to individuals who defy them. Model failure is reliable, phenotype specific and generalizable across datasets. Together, these results highlight the pitfalls of a one-size-fits-all modelling approach and the effect of biased phenotypic measures18-20 on the interpretation and utility of resulting brain-phenotype models. We present a framework to address these issues so that such models may reveal the neural circuits that underlie specific phenotypes and ultimately identify individualized neural targets for clinical intervention.

Proper understanding of recurrent stress urinary incontinence treatment in women (PURSUIT): a randomised controlled trial of endoscopic and surgical treatment

BACKGROUND

Women with stress urinary incontinence (SUI) experience urine leakage with physical activity. Currently, the interventional treatments for SUI are surgical, or endoscopic bulking injection(s). However, these procedures are not always successful, and symptoms can persist or come back after treatment, categorised as recurrent SUI. There are longstanding symptoms and distress associated with a failed primary treatment, and currently, there is no consensus on how best to treat women with recurrent, or persistent, SUI.

METHODS

A two-arm trial, set in at least 20 National Health Service (NHS) urology and urogynaecology referral units in the UK, randomising 250 adult women with recurrent or persistent SUI 1:1 to receive either an endoscopic intervention (endoscopic bulking injections) or a standard NHS surgical intervention, currently colposuspension, autologous fascial sling or artificial urinary sphincter. The aim of the trial is to determine whether surgical treatment is superior to endoscopic bulking injections in terms of symptom severity at 1 year after randomisation. This primary outcome will be measured using the patient-reported International Consultation on Incontinence Questionnaire - Urinary Incontinence - Short Form (ICIQ-UI-SF). Secondary outcomes include assessment of longer-term clinical impact, improvement of symptoms, safety, operative assessments, sexual function, cost-effectiveness and an evaluation of patients' and clinicians' views and experiences of the interventions.

DISCUSSION

There is a lack of high-quality, randomised, scientific evidence for which treatment is best for women presenting with recurrent SUI. The PURSUIT study will benefit healthcare professionals and patients and provide robust evidence to guide further treatment and improve symptoms and quality of life for women with this condition.

TRIAL REGISTRATION

International Standard Randomised Controlled Trials Number (ISRCTN) registry ISRCTN12201059. Registered on 09 January 2020.

Predicting the future of neuroimaging predictive models in mental health

Predictive modeling using neuroimaging data has the potential to improve our understanding of the neurobiology underlying psychiatric disorders and putatively information interventions. Accordingly, there is a plethora of literature reviewing published studies, the mathematics underlying machine learning, and the best practices for using these approaches. As our knowledge of mental health and machine learning continue to evolve, we instead aim to look forward and "predict" topics that we believe will be important in current and future studies. Some of the most discussed topics in machine learning, such as bias and fairness, the handling of dirty data, and interpretable models, may be less familiar to the broader community using neuroimaging-based predictive modeling in psychiatry. In a similar vein, transdiagnostic research and targeting brain-based features for psychiatric intervention are modern topics in psychiatry that predictive models are well-suited to tackle. In this work, we target an audience who is a researcher familiar with the fundamental procedures of machine learning and who wishes to increase their knowledge of ongoing topics in the field. We aim to accelerate the utility and applications of neuroimaging-based predictive models for psychiatric research by highlighting and considering these topics. Furthermore, though not a focus, these ideas generalize to neuroimaging-based predictive modeling in other clinical neurosciences and predictive modeling with different data types (e.g., digital health data).

Large-scale differences in functional organization of left- and right-handed individuals using whole-brain, data-driven analysis of connectivity

Handedness influences differences in lateralization of language areas as well as dominance of motor and somatosensory cortices. However, differences in whole-brain functional connectivity (i.e., functional connectomes) due to handedness have been relatively understudied beyond pre-specified networks of interest. Here, we compared functional connectomes of left- and right-handed individuals at the whole brain level. We explored differences in functional connectivity of previously established regions of interest, and showed differences between primarily left- and primarily right-handed individuals in the motor, somatosensory, and language areas using functional connectivity. We then proceeded to investigate these differences in the whole brain and found that the functional connectivity of left- and right-handed individuals are not specific to networks of interest, but extend across every region of the brain. In particular, we found that connections between and within the cerebellum show distinct patterns of connectivity. To put these effects into context, we show that the effect sizes associated with handedness differences account for a similar amount of individual differences in the connectome as sex differences. Together these results shed light on regions of the brain beyond those traditionally explored that contribute to differences in the functional organization of left- and right-handed individuals and underscore that handedness effects are neurobiologically meaningful in addition to being statistically significant.

A protocol for working with open-source neuroimaging datasets

Large, publicly available neuroimaging datasets are becoming increasingly common, but their use presents challenges because of insufficient knowledge of the tool options for data processing and proper data organization. Here, we describe a protocol to lessen these barriers. We describe the steps for the search and download of the open-source dataset. We detail the steps for proper data management and practical guidelines for data analysis. Finally, we give instructions for data and result sharing on public repositories and preprint services. For complete details on the use and execution of this profile, please refer to Horien et al. (2021).

Cetuximab and Irinotecan With or Without Bevacizumab in Refractory Metastatic Colorectal Cancer: BOND-3, an ACCRU Network Randomized Clinical Trial

Background

Combination irinotecan and cetuximab is approved for irinotecan-refractory metastatic colorectal cancer (mCRC). It is unknown if adding bevacizumab improves outcomes.

Patients and Methods

In this multicenter, randomized, double-blind, placebo-controlled phase II trial, patients with irinotecan-refractory RAS-wildtype mCRC and no prior anti-EGFR therapy were randomized to cetuximab 500 mg/m², bevacizumab 5 mg/kg, and irinotecan 180 mg/m² (or previously tolerated dose) (CBI) versus cetuximab, irinotecan, and placebo (CI) every 2 weeks until disease progression or intolerable toxicity. The primary endpoint was progression-free survival (PFS). Secondary endpoints included overall survival (OS), objective response rate (ORR), and adverse events (AEs).

Results

The study closed early after the accrual of 36 out of a planned 120 patients due to changes in funding. Nineteen patients were randomized to CBI and 17 to CI. Baseline characteristics were similar between arms. Median PFS was 9.7 versus 5.5 months for CBI and CI, respectively (1-sided log-rank P = .38; adjusted hazard ratio [HR] = 0.64; 95% confidence interval [CI], 0.25-1.66). Median OS was 19.7 versus 10.2 months for CBI and CI (1-sided log-rank P = .02; adjusted HR = 0.41; 95% CI, 0.15-1.09). ORR was 36.8% for CBI versus 11.8% for CI (P = .13). Grade 3 or higher AEs occurred in 47% of patients receiving CBI versus 35% for CI (P = .46).

Conclusion

In this prematurely discontinued trial, there was no significant difference in the primary endpoint of PFS between CBI and CI. There was a statistically significant improvement in OS in favor of CBI compared with CI. Further investigation of CBI for the treatment of irinotecan-refractory mCRC is warranted.

Clinical Trial Registration: NCT02292758

Abstract P2-09-19: Inequitable access to genetic testing leads to missed screening and prevention opportunities for individuals at risk for hereditary breast and ovarian cancer

Background: The National Comprehensive Cancer Network guideline on Genetic/Familial High Risk: Breast, Ovarian and Pancreatic Cancer (NCCN) recommends offering increased screening and risk-reduction options to women with a pathogenic/likely pathogenic variant (PV/LPV) in 8 clinically-actionable hereditary breast and ovarian cancer (HBOC) genes: ATM, BRCA1, BRCA2, CDH1, CHEK2, PALB2, PTEN, and TP53. These options range from biannual clinical breast exams and annual breast MRI screening to risk-reducing mastectomy (RRM) and salpingo-oophorectomy (RRSO). Yet health insurance company policies addressing the medical necessity of genetic testing for these genes vary widely. While some payers have adopted the NCCN testing criteria verbatim, others have made specific changes or have developed their own criteria. We sought to quantify the potential impact of this variability on access to enhanced medical management for carriers of PV/LPV in these 8 HBOC genes in a large cohort of patients undergoing genetic testing at a single commercial laboratory. Methods: We reviewed clinical and family histories for patients undergoing multigene panel testing that included testing for the 8 genes of interest and identified 107,344 patients who met NCCN testing criteria (excluding prostate cancer criteria, which were not assessed in this study) for HBOC (v.2.2021). We then compared the histories of these patients to testing criteria for 3 different payer groups: Aetna, Blue Shield of California/Federal Blue Cross-Blue Shield, and eviCore (used by over 30 payers including AmeriHealth, Highmark and Horizon). These are the largest payer groups in our cohort and together represent 19% of patients tested. For each patient we determined whether they met testing criteria for the 3 payer groups. For individuals found to have a PV/LPV we determined the potential missed management opportunities for those who would not have been tested under each payer’s criteria. In addition, we sought to estimate the maximal impact of this missed management for patients with BRCA PV/LPV (not accounting for patient age or medical history). Results: Among patients meeting NCCN testing criteria for HBOC, 10,10,477 (9.8%) were found to have PVs/LPVs in one of the 8 genes. Under the three payer policies 2% to 10.3% of all patients, and up to 4.0% (n=423) of PV/LPV carriers, would not have been eligible for genetic testing. Based on NCCN management guidelines, up to 423 eligible patients would not have been offered annual breast MRI, 208 patients would not have been offered RRM, and 163 eligible patients would not have been offered RRSO. This lack of testing access due to misaligned medical policies also represents missed opportunities for offering potentially life-saving screening and risk reduction measures to these patients as well. Assuming lifetime cancer risks of 85% for breast cancer and 40% for ovarian cancer, and risk-reduction of 95% with RRM and 80% with RRSO. Had all policies matched NCCN testing criteria, up to 132 breast cancers could have potentially been detected earlier or prevented and 52 ovarian cancers could potentially have been prevented in our cohort. Conclusions: In addition to complicating clinical practice, varying testing guidelines have broader implications. Our data show that a significant number of mutation carriers are being missed by payer policies that deviate from NCCN testing criteria. In turn, this represents missed opportunities to offer proactive screening and risk-reduction options that could potentially save lives for those at risk for hereditary cancer. Based on findings from other studies, these actions would likely reduce health insurer costs as well.

Citation Format: Robert T Pilarski, Stephanie Noble, Lily Hoang, Randy Hew, Stephanie Gandomi, Holly LaDuca, Jill Dolinsky. Inequitable access to genetic testing leads to missed screening and prevention opportunities for individuals at risk for hereditary breast and ovarian cancer [abstract]. In: Proceedings of the 2021 San Antonio Breast Cancer Symposium; 2021 Dec 7-10; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2022;82(4 Suppl):Abstract nr P2-09-19.

Patent foramen ovale closure: a comparison between two types of devices in a retrospective cohort

Background

Patent foramen ovale (PFO) closure has been shown to reduce the risk of recurrent stroke in selected patients. Different PFO occluding devices have been validated by the authorities and are used in the clinical practice.

Purpose

The main purpose of this study is to compare the use of two types of devices for PFO closure, the GORE CARDIOFORM Septal occluder and the AMPLATZER occluders (PFO or CRIBIFORM) in terms of major clinical safety and efficacy outcomes.

Methods

In this single-center retrospective study, the final cohort comprised 182 patients who underwent PFO closure between January 2012 and May 2019 (mean age 57±13 years, 34% males). The population was divided in two groups according to the device used (GORE CARDIOFORM Septal occluder [n=66, 36%] or the AMPLATZER occluders [n=116, 64%]). The endpoints assessed at 6 months, were the presence of a significant shunt (at least moderate [10–25 microbubbles] at rest or during the Valsalva manoeuvre), the incidence of supraventricular arrythmias, recurrent stroke, or serious device-related adverse events.

Results

At 6 months, a small percentage of patients with significant residual shunt was found in both the GORE and the AMPLATZER groups (7% vs 14% respectively, p=0.182). The incidence of supraventricular arrythmias was lower in the GORE as compared to the AMPLATZER group (9.1% vs 22.4% respectively, p=0.023). During the 6-month follow-up period there were 3 ischemic strokes, all of them in the AMPLATZER group, but this difference did not reach statistical significance (p=0.188). Finally, no serious device-related adverse events were noted in the total cohort.

Conclusion

PFO closure with a GORE device is related to lower incidence of supraventricular arrythmias at 6 months. The presence of a significant residual shunt and the incidence of recurrent stroke were comparable between both groups.

Funding Acknowledgement

Type of funding sources: None.

The instability of functional connectomes across the first year of life

The uniqueness and stability of the adolescent and adult functional connectome has been demonstrated to be high (80-95 % identification) using connectome-based identification (ID) or "fingerprinting". However, it is unclear to what extent individuals exhibit similar distinctiveness and stability in infancy, a developmental period of rapid and unparalleled brain development. In this study, we examined connectome-based ID rates within and across the first year of life using a longitudinal infant dataset at 1.5 month and 9 months of age. We also calculated the test-retest reliability of individual connections across the first year of life using the intraclass correlation coefficient (ICC). Overall, we found substantially lower infant ID rates than have been reported in adult and adolescent populations. Within-session ID rates were moderate and significant (ID = 48.94-70.83 %). Between-session ID rates were very low and not significant, with task-to-task connectomes resulting in the highest between-session ID rate (ID = 26.6 %). Similarly, average edge-level test-retest reliability was higher within-session than between-session (mean within-session ICC = 0.17, mean between-session ICC = 0.10). These findings suggest a lack of uniqueness and stability in functional connectomes across the first year of life consistent with the unparalleled changes in brain functional organization during this critical period.

Eliminating accidental deviations to minimize generalization error and maximize replicability: Applications in connectomics and genomics

Replicability, the ability to replicate scientific findings, is a prerequisite for scientific discovery and clinical utility. Troublingly, we are in the midst of a replicability crisis. A key to replicability is that multiple measurements of the same item (e.g., experimental sample or clinical participant) under fixed experimental constraints are relatively similar to one another. Thus, statistics that quantify the relative contributions of accidental deviations-such as measurement error-as compared to systematic deviations-such as individual differences-are critical. We demonstrate that existing replicability statistics, such as intra-class correlation coefficient and fingerprinting, fail to adequately differentiate between accidental and systematic deviations in very simple settings. We therefore propose a novel statistic, discriminability, which quantifies the degree to which an individual's samples are relatively similar to one another, without restricting the data to be univariate, Gaussian, or even Euclidean. Using this statistic, we introduce the possibility of optimizing experimental design via increasing discriminability and prove that optimizing discriminability improves performance bounds in subsequent inference tasks. In extensive simulated and real datasets (focusing on brain imaging and demonstrating on genomics), only optimizing data discriminability improves performance on all subsequent inference tasks for each dataset. We therefore suggest that designing experiments and analyses to optimize discriminability may be a crucial step in solving the replicability crisis, and more generally, mitigating accidental measurement error.

Centering inclusivity in the design of online conferences-An OHBM-Open Science perspective

As the global health crisis unfolded, many academic conferences moved online in 2020. This move has been hailed as a positive step towards inclusivity in its attenuation of economic, physical, and legal barriers and effectively enabled many individuals from groups that have traditionally been underrepresented to join and participate. A number of studies have outlined how moving online made it possible to gather a more global community and has increased opportunities for individuals with various constraints, e.g., caregiving responsibilities. Yet, the mere existence of online conferences is no guarantee that everyone can attend and participate meaningfully. In fact, many elements of an online conference are still significant barriers to truly diverse participation: the tools used can be inaccessible for some individuals; the scheduling choices can favour some geographical locations; the set-up of the conference can provide more visibility to well-established researchers and reduce opportunities for early-career researchers. While acknowledging the benefits of an online setting, especially for individuals who have traditionally been underrepresented or excluded, we recognize that fostering social justice requires inclusivity to actively be centered in every aspect of online conference design. Here, we draw from the literature and from our own experiences to identify practices that purposefully encourage a diverse community to attend, participate in, and lead online conferences. Reflecting on how to design more inclusive online events is especially important as multiple scientific organizations have announced that they will continue offering an online version of their event when in-person conferences can resume.

A guide to the measurement and interpretation of fMRI test-retest reliability

The test-retest reliability of functional neuroimaging data has recently been a topic of much discussion. Despite early conflicting reports, converging reports now suggest that test-retest reliability is poor for standard univariate measures-namely, voxel- and region-level task-based activation and edge-level functional connectivity. To better understand the implications of these recent studies requires understanding the nuances of test-retest reliability as commonly measured by the intraclass correlation coefficient (ICC). Here we provide a guide to the measurement and interpretation of test-retest reliability in functional neuroimaging and review major findings in the literature. We highlight the importance of making choices that improve reliability so long as they do not diminish validity, pointing to the potential of multivariate approaches that improve both. Finally, we discuss the implications of recent reports of low test-retest reliability in the context of ongoing work in the field.

Brainhack: Developing a culture of open, inclusive, community-driven neuroscience

Brainhack is an innovative meeting format that promotes scientific collaboration and education in an open, inclusive environment. This NeuroView describes the myriad benefits for participants and the research community and how Brainhacks complement conventional formats to augment scientific progress.

A hitchhiker's guide to working with large, open-source neuroimaging datasets

Large datasets that enable researchers to perform investigations with unprecedented rigor are growing increasingly common in neuroimaging. Due to the simultaneous increasing popularity of open science, these state-of-the-art datasets are more accessible than ever to researchers around the world. While analysis of these samples has pushed the field forward, they pose a new set of challenges that might cause difficulties for novice users. Here we offer practical tips for working with large datasets from the end-user's perspective. We cover all aspects of the data lifecycle: from what to consider when downloading and storing the data to tips on how to become acquainted with a dataset one did not collect and what to share when communicating results. This manuscript serves as a practical guide one can use when working with large neuroimaging datasets, thus dissolving barriers to scientific discovery.

Transdiagnostic, Connectome-Based Prediction of Memory Constructs Across Psychiatric Disorders

Memory deficits are observed in a range of psychiatric disorders, but it is unclear whether memory deficits arise from a shared brain correlate across disorders or from various dysfunctions unique to each disorder. Connectome-based predictive modeling is a computational method that captures individual differences in functional connectomes associated with behavioral phenotypes such as memory. We used publicly available task-based functional MRI data from patients with schizophrenia (n = 33), bipolar disorder (n = 34), attention deficit hyper-activity disorder (n = 32), and healthy controls (n = 73) to model the macroscale brain networks associated with working, short- and long-term memory. First, we use 10-fold and leave-group-out analyses to demonstrate that the same macroscale brain networks subserve memory across diagnostic groups and that individual differences in memory performance are related to individual differences within networks distributed throughout the brain, including the subcortex, default mode network, limbic network, and cerebellum. Next, we show that diagnostic groups are associated with significant differences in whole-brain functional connectivity that are distinct from the predictive models of memory. Finally, we show that models trained on the transdiagnostic sample generalize to novel, healthy participants (n = 515) from the Human Connectome Project. These results suggest that despite significant differences in whole-brain patterns of functional connectivity between diagnostic groups, the core macroscale brain networks that subserve memory are shared.

COVID-19 confessions: a qualitative exploration of healthcare workers experiences of working with COVID-19

OBJECTIVES

To gain insight into the experiences and concerns of front-line National Health Service (NHS) workers while caring for patients with COVID-19.

DESIGN

Qualitative analysis of data collected through an anonymous website (www.covidconfidential) provided a repository of uncensored COVID-19 experiences of front-line NHS workers, accessed via a link advertised on the Twitter feed of two high profile medical tweeters and their retweets.

SETTING

Community of NHS workers who accessed this social media.

PARTICIPANTS

54 healthcare workers, including doctors, nurses and physiotherapists, accessed the website and left a 'story'.

RESULTS

Stories ranged from 1 word to 10 min in length. Thematic analysis identified common themes, with a central aspect being the experience and psychological consequence of trauma. Specific themes were: (1) the shock of the virus, (2) staff sacrifice and dedication, (3) collateral damage ranging from personal health concerns to the long-term impact on, and care of, discharged patients and (4) a hierarchy of power and inequality within the healthcare system.

CONCLUSIONS

COVID-19 confidential gave an outlet for unprompted and uncensored stories of healthcare workers in the context of COVID-19. In addition to personal experiences of trauma, there were perceptions that many operational difficulties stemmed from inequalities of power between management and front-line workers. Learning from these experiences will reduce staff distress and improve patient care in the face of further waves of the pandemic.

Modelling the lifetime cost-effectiveness of radical prostatectomy, radiotherapy and active monitoring for men with clinically localised prostate cancer from median 10-year outcomes in the ProtecT randomised trial

BACKGROUND

Optimal management strategies for clinically localised prostate cancer are debated. Using median 10-year data from the largest randomised controlled trial to date (ProtecT), the lifetime cost-effectiveness of three major treatments (radical radiotherapy, radical prostatectomy and active monitoring) was explored according to age and risk subgroups.

METHODS

A decision-analytic (Markov) model was developed and informed by clinical input. The economic evaluation adopted a UK NHS perspective and the outcome was cost per Quality-Adjusted Life Year (QALY) gained (reported in UK£), estimated using EQ-5D-3L.

RESULTS

Costs and QALYs extrapolated over the lifetime were mostly similar between the three randomised strategies and their subgroups, but with some important differences. Across all analyses, active monitoring was associated with higher costs, probably associated with higher rates of metastatic disease and changes to radical treatments. When comparing the value of the strategies (QALY gains and costs) in monetary terms, for both low-risk prostate cancer subgroups, radiotherapy generated the greatest net monetary benefit (£293,446 [95% CI £282,811 to £299,451] by D'Amico and £292,736 [95% CI £284,074 to £297,719] by Grade group 1). However, the sensitivity analysis highlighted uncertainty in the finding when stratified by Grade group, as radiotherapy had 53% probability of cost-effectiveness and prostatectomy had 43%. In intermediate/high risk groups, using D'Amico and Grade group > = 2, prostatectomy generated the greatest net monetary benefit (£275,977 [95% CI £258,630 to £285,474] by D'Amico and £271,933 [95% CI £237,864 to £287,784] by Grade group). This finding was supported by the sensitivity analysis. Prostatectomy had the greatest net benefit (£290,487 [95% CI £280,781 to £296,281]) for men younger than 65 and radical radiotherapy (£201,311 [95% CI £195,161 to £205,049]) for men older than 65, but sensitivity analysis showed considerable uncertainty in both findings.

CONCLUSION

Over the lifetime, extrapolating from the ProtecT trial, radical radiotherapy and prostatectomy appeared to be cost-effective for low risk prostate cancer, and radical prostatectomy for intermediate/high risk prostate cancer, but there was uncertainty in some estimates. Longer ProtecT trial follow-up is required to reduce uncertainty in the model.

TRIAL REGISTRATION

Current Controlled Trials number, ISRCTN20141297: http://isrctn.org (14/10/2002); ClinicalTrials.gov number, NCT02044172: http://www.clinicaltrials.gov (23/01/2014).

The Constrained Network-Based Statistic: A New Level of Inference for Neuroimaging

Neuroimaging research aimed at dissecting the network organization of the brain is poised to flourish under major initiatives, but converging evidence suggests more accurate inferential procedures are needed to promote discovery. Inference is typically performed at the cluster level with a network-based statistic (NBS) that boosts power by leveraging known dependence within the local neighborhood. However, existing NBS methods overlook another important form of dependence-shared membership in large-scale brain networks. Here, we propose a new level of inference that pools information within predefined large-scale networks: the Constrained Network-Based Statistic (cNBS). We evaluated sensitivity and specificity of cNBS against existing standard NBS and threshold-free NBS by resampling task data from the largest openly available fMRI database: the Human Connectome Project. cNBS was most sensitive to effect sizes below medium, which accounts for the majority of ground truth effects. In contrast, threshold-free NBS was most sensitive to higher effect sizes. Ground truth maps showed grouping of effects within large-scale networks, supporting the relevance of cNBS. All methods controlled FWER as intended. In summary, cNBS is a promising new level of inference for promoting more valid inference, a critical step towards more reproducible discovery in neuroscience.

How Tasks Change Whole-Brain Functional Organization to Reveal Brain-Phenotype Relationships

Functional connectivity (FC) calculated from task fMRI data better reveals brain-phenotype relationships than rest-based FC, but how tasks have this effect is unknown. In over 700 individuals performing seven tasks, we use psychophysiological interaction (PPI) and predictive modeling analyses to demonstrate that task-induced changes in FC successfully predict phenotype, and these changes are not simply driven by task activation. Activation, however, is useful for prediction only if the in-scanner task is related to the predicted phenotype. To further characterize these predictive FC changes, we develop and apply an inter-subject PPI analysis. We find that moderate, but not high, task-induced consistency of the blood-oxygen-level-dependent (BOLD) signal across individuals is useful for prediction. Together, these findings demonstrate that in-scanner tasks have distributed, phenotypically relevant effects on brain functional organization, and they offer a framework to leverage both task activation and FC to reveal the neural bases of complex human traits, symptoms, and behaviors.

Cluster failure or power failure? Evaluating sensitivity in cluster-level inference

Pioneering work in human neuroscience has relied on the ability to map brain function using task-based fMRI, but the empirical validity of these inferential methods is still being characterized. A recent landmark study by Eklund and colleagues showed that popular multiple comparison corrections based on cluster extent suffer from unexpectedly low specificity (i.e., high false positive rate). Yet that study's focus on specificity, while important, is incomplete. The validity of a method depends also on its sensitivity (i.e., true positive rate or power), yet the sensitivity of cluster correction remains poorly understood. Here, we assessed the sensitivity of gold-standard nonparametric cluster correction by resampling real data from five tasks in the Human Connectome Project and comparing results with those from the full "ground truth" datasets (n ​= ​480-493). Critically, we found that sensitivity after correction is lower than may be practical for many fMRI applications. In particular, sensitivity to medium-sized effects (|Cohen's d| ​= ​0.5) was less than 20% across tasks on average, about three times smaller than without any correction. Furthermore, cluster extent correction exhibited a spatial bias in sensitivity that was independent of effect size. In comparison, correction based on the Threshold-Free Cluster Enhancement (TFCE) statistic approximately doubled sensitivity across tasks but increased spatial bias. These results suggest that we have, until now, only measured the tip of the iceberg in the activation-mapping literature due to our goal of limiting the familywise error rate through cluster extent-based inference. There is a need to revise our practices to improve sensitivity; we therefore conclude with a list of modern strategies to boost sensitivity while maintaining respectable specificity in future investigations.

Measuring functional limitations after venous thromboembolism: Optimization of the Post-VTE Functional Status (PVFS) Scale

INTRODUCTION

We recently proposed a scale for assessment of patient-relevant functional limitations following an episode of venous thromboembolism (VTE). Further development of this post-VTE functional status (PVFS) scale is still needed.

METHODS

Guided by the input of VTE experts and patients, we refined the PVFS scale and its accompanying manual, and attempted to acquire broad consensus on its use.

RESULTS

A Delphi analysis was performed involving 53 international VTE experts with diverse scientific and clinical backgrounds. In this process, the number of scale grades of the originally proposed PVFS scale was reduced and descriptions of the grades were improved. After these changes, a consensus was reached on the number/definitions of the grades, and method/timing of the scale assessment. The relevance and potential impact of the scale was confirmed in three focus groups totaling 18 VTE patients, who suggested additional changes to the manual, but not to the scale itself. Using the improved manual, the κ-statistics between PVFS scale self-reporting and its assessment via the structured interview was 0.75 (95%CI 0.58-1.0), and 1.0 (95%CI 0.83-1.0) between independent raters of the recorded interview of 16 focus groups members.

CONCLUSION

We improved the PVFS scale and demonstrated broad consensus on its relevance, optimal grades, and methods of assessing among international VTE experts and patients. The interobserver agreement of scale grade assignment was shown to be good-to-excellent. The PVFS scale may become an important outcome measure of functional impairment for quality of patient care and in future VTE trials.

Irinotecan, cetuximab, and bevacizumab (CBI) versus irinotecan, cetuximab, and placebo (CI) in irinotecan-refractory metastatic colorectal cancer (mCRC): Results from an ACCRU network randomized phase II trial

102

Background: Combination irinotecan and cetuximab is approved for irinotecan-refractory mCRC; it is unknown if the addition of bevacizumab would improve outcomes. We studied the efficacy and safety of CBI compared with CI in patients (pts) with RAS wildtype, irinotecan-refractory mCRC. Methods: In this multicenter, randomized, double-blind, placebo-controlled phase II trial, pts with RAS wildtype mCRC and no prior anti-epidermal growth factor receptor therapy who failed at least 1 irinotecan-based chemotherapy regimen and received bevacizumab in at least 1 prior line of therapy were randomized 1:1 to irinotecan 180 mg/m2 (or previously tolerated dose), cetuximab 500 mg/m2, and bevacizumab 5 mg/kg vs CI every 2 wks until disease progression, intolerable toxicity, or withdrawal of consent. The primary endpoint was progression free survival (PFS). Multivariable Cox proportional hazard models stratified by number of prior lines of therapy and bevacizumab receipt in immediate prior line were performed. Secondary endpoints included overall survival (OS), objective response rate (ORR), and adverse events (AEs). The study was closed early in January 2018 for reasons related to accrual and funding after enrollment of 36 out of a planned 60 pts. Results: Between July 2015 and December 2017, 36 pts were randomized (19 to CBI, 17 to CI). 34 pts (94%) were treated with 2 or more prior chemotherapy regimens. Baseline characteristics were similar between arms. Median PFS was 9.7 vs 5.5 mo for CBI and CI arms, respectively (log-rank P =0.76; multivariable HR = 0.64; 95% CI, 0.25-1.66). Median OS was 19.7 vs 10.2 mo for CBI and CI (log-rank P= 0.04; multivariable HR = 0.41; 95% CI, 0.15-1.09). ORR was 37% for CBI vs 12% for CI ( P =0.13). Grade 3 or higher AEs occurred in 47% of pts receiving CBI vs 35% for CI ( P =0.46). Conclusions: In this prematurely discontinued trial, there were non-significant increases in PFS and ORR and a statistically significant 9.5 mo increase in median OS in favor of CBI compared to CI. Further investigation of CBI for treatment of irinotecan-refractory mCRC is warranted. Clinical trial information: NCT02292758.

A decade of test-retest reliability of functional connectivity: A systematic review and meta-analysis

BACKGROUND

Once considered mere noise, fMRI-based functional connectivity has become a major neuroscience tool in part due to early studies demonstrating its reliability. These fundamental studies revealed only the tip of the iceberg; over the past decade, many test-retest reliability studies have continued to add nuance to our understanding of this complex topic. A summary of these diverse and at times contradictory perspectives is needed.

OBJECTIVES

We aimed to summarize the existing knowledge regarding test-retest reliability of functional connectivity at the most basic unit of analysis: the individual edge level. This entailed (1) a meta-analytic estimate of reliability and (2) a review of factors influencing reliability.

METHODS

A search of Scopus was conducted to identify studies that estimated edge-level test-retest reliability. To facilitate comparisons across studies, eligibility was restricted to studies measuring reliability via the intraclass correlation coefficient (ICC). The meta-analysis included a random effects pooled estimate of mean edge-level ICC, with studies nested within datasets. The review included a narrative summary of factors influencing edge-level ICC.

RESULTS

From an initial pool of 212 studies, 44 studies were identified for the qualitative review and 25 studies for quantitative meta-analysis. On average, individual edges exhibited a "poor" ICC of 0.29 (95% CI = 0.23 to 0.36). The most reliable measurements tended to involve: (1) stronger, within-network, cortical edges, (2) eyes open, awake, and active recordings, (3) more within-subject data, (4) shorter test-retest intervals, (5) no artifact correction (likely due in part to reliable artifact), and (6) full correlation-based connectivity with shrinkage.

CONCLUSION

This study represents the first meta-analysis and systematic review investigating test-retest reliability of edge-level functional connectivity. Key findings suggest there is room for improvement, but care should be taken to avoid promoting reliability at the expense of validity. By pooling existing knowledge regarding this key facet of accuracy, this study supports broader efforts to improve inferences in the field.

P3630Coronary angiography and percutaneous coronary intervention post implantation of a transcatheter aortic valve

When treating younger patients with longer life expectancy, the likelihood of having to perform a subsequent coronary angiogram and PCI during follow-up post transcatheter aortic valve implantation (TAVI) is increased. The aim of our study was to assess the feasibility and characteristics of the post TAVI coronary angiogram.

We interrogated our prospective single center TAVI database to analyze data from patients who underwent a coronary angiogram or PCI post TAVI. Between August 2008 and January 2019, 405 consecutive TAVI were performed in our institution with a 30-day, 6-month and yearly follow-up.

Among 405 procedures, 18 coronary angiograms with 9 PCI (RCA 2; LM 3; LAD 3, LCX 1) were performed in 17 patients post TAVI. The mean age of patients was 78.1±7.5 years (3 women), with a mean STS score of 5.6±3.4%. Five patients had a history of prior CABG. The transcatheter heart valves implanted were 6 CoreValve, 8 Evolut R, 1 Evolut Pro and 2 Edwards Sapien. The valve sizes were 31mm (n=3), 29mm (n=6), 26mm (n=3) and 23mm (n=5). The indication for the coronary angiogram was acute coronary syndrome for 8 procedures (1 STEMI, 7 NSTEMI) and stable angina for the remaining 10 procedures. The mean time between TAVI and coronary angiogram was 519±332 days (189–1093 days). Femoral and radial approach was used in, respectively, 11 and 7 (right: 5, left: 2) procedures with one cross-over needed from right transradial to transfemoral. Out of 33 attempted ostia cannulations, there were 2 failures (both involving the RCA), and 31 successes classified as selective (RCA: 7, LM: 9), borderline selective (RCA: 3, LM: 2) and non-selective but sufficient for diagnosis (RCA: 3, LM: 7), using standard catheters (average number of catheter used to cannulate RCA: 1.2±0.8 and LM: 1.7±1.1 [min. 1 and max. 4]). All planned PCI were successful with the treatment of 11 stenosis including 2 multivessel lesions. In 44% of the treated arteries, the use of a GuideLiner catheter (Vascular Solutions Inc) was required to cannulate the ostium.

In conclusion, coronary angiogram post TAVI was needed in 4.2% of our cohort during follow-up. Overall, LM ostia cannulation was successful in 100% of the cases and we failed to cannulate 13.3% of RCA ostia. All PCI were successful with the need of a guiding cathteter extension in 44% of the procedures.

Multivariate approaches improve the reliability and validity of functional connectivity and prediction of individual behaviors

Brain functional connectivity features can predict cognition and behavior at the level of the individual. Most studies measure univariate signals, correlating timecourses from the average of constituent voxels in each node. While straightforward, this approach overlooks the spatial patterns of voxel-wise signals within individual nodes. Given that multivariate spatial activity patterns across voxels can improve fMRI measures of mental representations, here we asked whether using voxel-wise timecourses can better characterize region-by-region interactions relative to univariate approaches. Using two fMRI datasets, the Human Connectome Project sample and a local test-retest sample, we measured multivariate functional connectivity with multivariate distance correlation and univariate connectivity with Pearson's correlation. We compared multivariate and univariate connectivity estimates, demonstrating that relative to univariate estimates, multivariate estimates exhibited higher reliability at both the edge-level and connectome-level, stronger prediction of individual differences, and greater sensitivity to brain states within individuals. Our findings suggest that multivariate estimates reliably provide more powerful information about an individual's functional brain organization and its relation to cognitive skills.