Generalized reliability based on distances

Reliability of landmark identification for analysis of the temporomandibular joint in real-time MRI

BACKGROUND

Real-time magnetic resonance imaging (rtMRI) is essential for diagnosing and comprehending temporomandibular joint (TMJ) movements. Current methods for tracking and analysis require manual landmark placement on each acquisition frame. Therefore, our study aimed to assess the inter- and intra-rater reliability of placing cephalometric landmarks in frames from a dynamic real-time TMJ MRI.

MATERIAL AND METHODS

Four real-time MRIs of the right TMJ were taken during mandibular movement at ten frames per second. Seven dentists identified ten landmarks on two frames (intercuspal position-ICP-and maximum mouth opening-MMO) twice at a two-week interval, yielding 112 tracings. Six typical cephalometric measurements (angles and distances) were derived from these landmarks. The reliabilities of landmarks and measurements were evaluated using distance-based (dbICC), linear mixed effect model intraclass correlation (lmeICC), and standard ICC.

RESULTS

The average inter-rater reliability for the landmarks stood at 0.92 (dbICC) and 0.93 (lmeICC). The intra-rater reliability scores were 0.97 and 0.98. Over 80% of the landmarks showed an ICC greater than 0.98 (inter-rater) and over 0.99 (intra-rater). The lowest landmark ICC was observed for the orbitale and the oblique ridge of the mandibular ramus. However, the cephalometric angle and distance measurements derived from these landmarks showed only moderate to good reliability, whereas the reliability in the frames with ICP was better than those with MMO. Measurements performed in the ICP frame were more reliable than measurements in the MMO frame.

CONCLUSION

While dentists reliably localize isolated landmarks in real-time MRIs, the cephalometric measurements derived from them remain inconsistent. The better results in ICP than MMO are probably due to a more familiar jaw position. The higher error rate of the TMJ measurements in MMO could be associated with a lack of training in real-time MRI analysis in dentistry.

ReX: an integrative tool for quantifying and optimizing measurement reliability for the study of individual differences

Characterizing multifaceted individual differences in brain function using neuroimaging is central to biomarker discovery in neuroscience. We provide an integrative toolbox, Reliability eXplorer (ReX), to facilitate the examination of individual variation and reliability as well as the effective direction for optimization of measuring individual differences in biomarker discovery. We also illustrate gradient flows, a two-dimensional field map-based approach to identifying and representing the most effective direction for optimization when measuring individual differences, which is implemented in ReX.

PhiPipe: A multi-modal MRI data processing pipeline with test-retest reliability and predicative validity assessments

Magnetic resonance imaging (MRI) has been one of the primary instruments to measure the properties of the human brain non-invasively in vivo. MRI data generally needs to go through a series of processing steps (i.e., a pipeline) before statistical analysis. Currently, the processing pipelines for multi-modal MRI data are still rare, in contrast to single-modal pipelines. Furthermore, the reliability and validity of the output of the pipelines are critical for the MRI studies. However, the reliability and validity measures are not available or adequate for almost all pipelines. Here, we present PhiPipe, a multi-modal MRI processing pipeline. PhiPipe could process T1-weighted, resting-state BOLD, and diffusion-weighted MRI data and generate commonly used brain features in neuroimaging. We evaluated the test-retest reliability of PhiPipe's brain features by computing intra-class correlations (ICC) in four public datasets with repeated scans. We further evaluated the predictive validity by computing the correlation of brain features with chronological age in three public adult lifespan datasets. The multivariate reliability and predictive validity of the PhiPipe results were also evaluated. The results of PhiPipe were consistent with previous studies, showing comparable or better reliability and validity when compared with two popular single-modality pipelines, namely DPARSF and PANDA. The publicly available PhiPipe provides a simple-to-use solution to multi-modal MRI data processing. The accompanied reliability and validity assessments could help researchers make informed choices in experimental design and statistical analysis. Furthermore, this study provides a framework for evaluating the reliability and validity of image processing pipelines.

Generalized reliability based on distances

The intraclass correlation coefficient (ICC) is a classical index of measurement reliability. With the advent of new and complex types of data for which the ICC is not defined, there is a need for new ways to assess reliability. To meet this need, we propose a new distance-based ICC (dbICC), defined in terms of arbitrary distances among observations. We introduce a bias correction to improve the coverage of bootstrap confidence intervals for the dbICC, and demonstrate its efficacy via simulation. We illustrate the proposed method by analyzing the test-retest reliability of brain connectivity matrices derived from a set of repeated functional magnetic resonance imaging scans. The Spearman-Brown formula, which shows how more intensive measurement increases reliability, is extended to encompass the dbICC.