Planning stepped wedge cluster randomized trials to detect treatment effect heterogeneity

Stepped wedge design is a popular research design that enables a rigorous evaluation of candidate interventions by using a staggered cluster randomization strategy. While analytical methods were developed for designing stepped wedge trials, the prior focus has been solely on testing for the average treatment effect. With a growing interest on formal evaluation of the heterogeneity of treatment effects across patient subpopulations, trial planning efforts need appropriate methods to accurately identify sample sizes or design configurations that can generate evidence for both the average treatment effect and variations in subgroup treatment effects. To fill in that important gap, this article derives novel variance formulas for confirmatory analyses of treatment effect heterogeneity, that are applicable to both cross-sectional and closed-cohort stepped wedge designs. We additionally point out that the same framework can be used for more efficient average treatment effect analyses via covariate adjustment, and allows the use of familiar power formulas for average treatment effect analyses to proceed. Our results further sheds light on optimal design allocations of clusters to maximize the weighted precision for assessing both the average and heterogeneous treatment effects. We apply the new methods to the Lumbar Imaging with Reporting of Epidemiology Trial, and carry out a simulation study to validate our new methods.

Short-term and long-term phenotypic stability of actigraphic sleep metrics involving repeated sleep loss and recovery

For the first time, we determined whether actigraphic-assessed sleep measures show inter-individual differences and intra-individual stability during baseline (BL) and recovery (REC) phases surrounding repeated total sleep deprivation (TSD). We conducted a 5-day experiment at Months 2 and 4 in two separate studies (N = 11). During each experiment, sleep measures were collected via wrist actigraphy during two BL 8 h time-in-bed (TIB) nights (B1, B2) and during two REC 8-10 h TIB nights (R1, R2). Intraclass correlation coefficients (ICCs) assessed actigraphic measure long-term stability between 2 and 4 months for (1) the pre-experimental phase before BL; and (2) the BL (B1 + B2), REC (R1 + R2), and BL and REC average (BL + REC) phases; and short-term stability at Month 2 and at Month 4; and (3) between B1 versus B2 and R1 versus R2 in each 5-day experiment. Nearly all ICCs during the pre-experimental, BL, REC, and BL + REC phases were moderate to almost perfect (0.446-0.970) between Months 2 and 4. B1 versus B2 ICCs were more stable (0.440-0.899) than almost all R1 versus R2 ICCs (-0.696 to 0.588) at Month 2 and 4. Actigraphic sleep measures show phenotypic long-term stability during BL and REC surrounding repeated TSD between 2 and 4 months. Furthermore, within each 5-day experiment at Month 2 and 4, the two BL nights before TSD were more stable than the two REC nights following TSD, likely due to increased R1 homeostatic pressure. Given the consistency of actigraphic measures across the short-term and long-term, they can serve as biomarkers to predict physiological and neurobehavioral responses to sleep loss.

Assessing intra- and inter-method agreement of functional data

Modern medical devices are increasingly producing complex data that could offer deeper insights into physiological mechanisms of underlying diseases. One type of complex data that arises frequently in medical imaging studies is functional data, whose sampling unit is a smooth continuous function. In this work, with the goal of establishing the scientific validity of experiments involving modern medical imaging devices, we focus on the problem of evaluating reliability and reproducibility of multiple functional data that are measured on the same subjects by different methods (i.e. different technologies or raters). Specifically, we develop a series of intraclass correlation coefficient and concordance correlation coefficient indices that can assess intra-method, inter-method, and total (intra + inter) agreement based on multivariate multilevel functional data consisting of replicated functional data measurements produced by each of the different methods. For efficient estimation, the proposed indices are expressed using variance components of a multivariate multilevel functional mixed effect model, which can be smoothly estimated by functional principal component analysis. Extensive simulation studies are performed to assess the finite-sample properties of the estimators. The proposed method is applied to evaluate the reliability and reproducibility of renogram curves produced by a high-tech radionuclide image scan used to non-invasively detect kidney obstruction.

Designing multicenter individually randomized group treatment trials

In an individually randomized group treatment (IRGT) trial, participant outcomes can be positively correlated due to, for example, shared therapists in treatment delivery. Oftentimes, because of limited treatment resources or participants at one location, an IRGT trial can be carried out across multiple centers. This design can be subject to potential correlations in the participant outcomes between arms within the same center. While the design of a single-center IRGT trial has been studied, little is known about the planning of a multicenter IRGT trial. To address this gap, this paper provides analytical sample size formulas for designing multicenter IRGT trials with a continuous endpoint under the linear mixed model framework. We found that accounting for the additional center-level correlation at the design stage can lead to sample size reduction, and the magnitude of reduction depends on the amount of between-therapist correlation. However, if the variance components of therapist-level random effects are considered as input parameters in the design stage, accounting for the additional center-level variance component has no impact on the sample size estimation. We presented our findings through numeric illustrations and performed simulation studies to validate our sample size procedures under different scenarios. Optimal design configurations under the multicenter IRGT trials have also been discussed, and two real-world trial examples are drawn to illustrate the use of our method.

Validity of the Entralpi force plate in the assessment of finger flexor performance metrics in rock climbers

This study assessed the validity of the Entralpi force plate in the assessment of finger flexor performance in rock climbers. In addition to a static force evaluation, peak force, peak impulse, and total impulse were measured during 30 all-out performance trials by 15 participants, in which force during the trials was recorded simultaneously by the Entralpi and a Pasco force plate. Agreement between devices was assessed by a variety of statistical analyses, including intraclass correlation coefficient (ICC), coefficient of variation (CV), and Bland-Altman analyses. The static force evaluation showed a mean relative error of 0.41% and excellent day-to-day reliability (ICC = 1; CV = 0.03%). Peak force, peak impulse, and total impulse from the performance trials demonstrated strong agreement (ICC ≥ 0.991, CV ≤ 1.9%, Bland-Altman mean bias ≤ 0.5%). These results illustrate that the Entralpi force plate provides accurate and reliable data for rock climbing related tasks at an affordable cost.

Cardiovascular measures display robust phenotypic stability across long-duration intervals involving repeated sleep deprivation and recovery

Introduction

We determined whether cardiovascular (CV) measures show trait-like responses after repeated total sleep deprivation (TSD), baseline (BL) and recovery (REC) exposures in two long-duration studies (total N = 11 adults).

Methods

A 5-day experiment was conducted twice at months 2 and 4 in a 4-month study (N = 6 healthy adults; 3 females; mean age ± SD, 34.3 ± 5.7 years; mean BMI ± SD, 22.5 ± 3.2 kg/m2), and three times at months 2, 4, and 8 in an 8-month study (N = 5 healthy adults; 2 females; mean age ± SD, 33.6 ± 5.17 years; mean BMI ± SD, 27.1 ± 4.9 kg/m2). Participants were not shift workers or exposed to TSD in their professions. During each experiment, various seated and standing CV measures were collected via echocardiography [stroke volume (SV), heart rate (HR), cardiac index (CI), left ventricular ejection time (LVET), and systemic vascular resistance index (SVRI)] or blood pressure monitor [systolic blood pressure (SBP)] after (1) two BL 8h time in bed (TIB) nights; (2) an acute TSD night; and (3) two REC 8-10 h TIB nights. Intraclass correlation coefficients (ICCs) assessed CV measure stability during BL, TSD, and REC and for the BL and REC average (BL + REC) across months 2, 4, and 8; Spearman's rho assessed the relative rank of individuals' CV responses across measures.

Results

Seated BL (0.693-0.944), TSD (0.643-0.962) and REC (0.735-0.960) CV ICCs showed substantial to almost perfect stability and seated BL + REC CV ICCs (0.552-0.965) showed moderate to almost perfect stability across months 2, 4, and 8. Individuals also exhibited significant, consistent responses within seated CV measures during BL, TSD, and REC. Standing CV measures showed similar ICCs for BL, TSD, and REC and similar response consistency.

Discussion

This is the first demonstration of remarkably robust phenotypic stability of a number of CV measures in healthy adults during repeated TSD, BL and REC exposures across 2, 4, and 8 months, with significant consistency of responses within CV measures. The cardiovascular measures examined in our studies, including SV, HR, CI, LVET, SVRI, and SBP, are useful biomarkers that effectively track physiology consistently across long durations and repeated sleep deprivation and recovery.

Studies on Reliability and Measurement Error of Measurements in Medicine - From Design to Statistics Explained for Medical Researchers

Reliability and measurement error are measurement properties that quantify the influence of specific sources of variation, such as raters, type of machine, or time, on the score of the individual measurement. Several designs can be chosen to assess reliability and measurement error of a measurement. Differences in design are due to specific choices about which sources of variation are varied over the repeated measurements in stable patients, which potential sources of variation are kept stable (ie, restricted), and about whether or not the entire measurement instrument (or measurement protocol) was repeated or only part of it. We explain how these choices determine how intraclass correlation coefficients and standard errors of measurement formulas are built for different designs by using Venn diagrams. Strategies for improving the measurement are explained, and recommendations for reporting the essentials of these studies are described. We hope that this paper will facilitate the understanding and improve the design, analysis, and reporting of future studies on reliability and measurement error of measurements.

Cook with Different Pots, but Similar Taste? Comparison of Phase Angle Using Bioelectrical Impedance Analysis According to Device Type and Examination Posture

Bioelectrical impedance analysis (BIA) is gaining popularity as a tool for body composition assessment. Although BIA has been studied and validated in different populations, age groups, and clinical settings, including critically ill patients, there are concerns about BIA reproducibility and reliability for different device types and postures. This study aimed to evaluate the reliability of BIA using different devices, postures, and lead types. Cross-sectional observational data were collected from 74 healthy volunteers (32 women, 42 men). We used two types of devices, three types of postures (standing, sitting, and lying), and two lead types (clamp lead and adhesive lead) to measure the whole-body phase angle (phA) at a single frequency of 50 kHz. The measurements were validated using the intraclass correlation coefficient (ICC) and Bland-Altman plot analysis. All phA measurements recorded using the two types of devices, three different postures, and two types of leads were equivalent (mean ICC = 0.9932, 95% confidence interval (CI) 0.9905-0.0053, p < 0.001). The average mean difference in phA was 0.31 (95% CI 0.16-0.46). The largest phA value was measured using BWA with an adhesive-type lead in the supine position. There were no differences between the standing and sitting positions. We compared the consistency and reliability of phA using two devices, two lead types, and three postures. Seven different phA were interchangeable in healthy volunteers.

Test-retest reliability of a finger-tapping fMRI task in a healthy population

Measuring brain activity during functional MRI (fMRI) tasks is one of the main tools to identify brain biomarkers of disease or neural substrates associated with specific symptoms. However, identifying correct biomarkers relies on reliable measures. Recently, poor reliability was reported for task-based fMRI measures. The present study aimed to demonstrate the reliability of a finger-tapping fMRI task across two sessions in healthy participants. Thirty-one right-handed healthy participants aged 18-60 years took part in two MRI sessions 3 weeks apart during which we acquired finger-tapping task-fMRI. We examined the overlap of activations between sessions using Dice similarity coefficients, assessing their location and extent. Then, we compared amplitudes calculating intraclass correlation coefficients (ICCs) in three sets of regions of interest (ROIs) in the motor network: literature-based ROIs (10-mm-radius spheres centred on peaks of an activation likelihood estimation), anatomical ROIs (regions as defined in an atlas) and ROIs based on conjunction analyses (superthreshold voxels in both sessions). Finger tapping consistently activated expected regions, for example, left primary sensorimotor cortices, premotor area and right cerebellum. We found good-to-excellent overlap of activations for most contrasts (Dice coefficients: .54-.82). Across time, ICCs showed large variability in all ROI sets (.04-.91). However, ICCs in most ROIs indicated fair-to-good reliability (mean = .52). The least specific contrast consistently yielded the best reliability. Overall, the finger-tapping task showed good spatial overlap and fair reliability of amplitudes on group level. Although caution is warranted in interpreting correlations of activations with other variables, identification of activated regions in response to a task and their between-group comparisons are still valid and important modes of analysis in neuroimaging to find population tendencies and differences.

Tracking cognition with the T-MoCA in a racially/ethnically diverse older adult cohort

Introduction

We investigated the utility of the Telephone-Montreal Cognitive Assessment (T-MoCA) to track cognition in a diverse sample from the Einstein Aging Study.

Methods

Telephone and in-person MoCA data, collected annually, were used to evaluate longitudinal cognitive performance. Joint models of T-MoCA and in-person MoCA compared changes, variance, and test-retest reliability measured by intraclass correlation coefficient by racial/ethnic group.

Results

There were no significant differences in baseline performance or longitudinal changes across three study waves for both MoCA formats. T-MoCA performance improved over waves 1-3 but declined afterward. Test-retest reliability was lower for the T-MoCA than for the in-person MoCA. In comparison with non-Hispanic Whites, non-Hispanic Blacks and Hispanics performed worse at baseline on both MoCA formats and showed lower correlations between T-MoCA and in-person versions.

Conclusions

The T-MoCA provides valuable information on cognitive change, despite racial/ethnic disparities and practice effects. We discuss implications for health disparity populations.

Highlights

We assessed the comparability of Telephone-Montreal Cognitive Assessment (T-MoCA) and in-person MoCA for tracking cognition.Changes within 3 years in T-MoCA were similar to that for the in-person MoCA.T-MoCA is subject to practice effects and shows difference in performance by race/ethnicity.Test-retest reliability of T-MoCA is lower than that for in-person MoCA.

Effects of Physiological Signal Removal on Resting-State Functional MRI Metrics

Resting-state fMRIs (rs-fMRIs) have been widely used for investigation of diverse brain functions, including brain cognition. The rs-fMRI has easily elucidated rs-fMRI metrics, such as the fractional amplitude of low-frequency fluctuation (fALFF), regional homogeneity (ReHo), voxel-mirrored homotopic connectivity (VMHC), and degree centrality (DC). To increase the applicability of these metrics, higher reliability is required by reducing confounders that are not related to the functional connectivity signal. Many previous studies already demonstrated the effects of physiological artifact removal from rs-fMRI data, but few have evaluated the effect on rs-fMRI metrics. In this study, we examined the effect of physiological noise correction on the most common rs-fMRI metrics. We calculated the intraclass correlation coefficient of repeated measurements on parcellated brain areas by applying physiological noise correction based on the RETROICOR method. Then, we evaluated the correction effect for five rs-fMRI metrics for the whole brain: FC, fALFF, ReHo, VMHC, and DC. The correction effect depended not only on the brain region, but also on the metric. Among the five metrics, the reliability in terms of the mean value of all ROIs was significantly improved for FC, but it deteriorated for fALFF, with no significant differences for ReHo, VMHC, and DC. Therefore, the decision on whether to perform the physiological correction should be based on the type of metric used.

Repeatability of the Vibroarthrogram in the Temporomandibular Joints

Current research concerning the repeatability of the joint's sounds examination in the temporomandibular joints (TMJ) is inconclusive; thus, the aim of this study was to investigate the repeatability of the specific features of the vibroarthrogram (VAG) in the TMJ using accelerometers. The joint sounds of both TMJs were measured with VAG accelerometers in two groups, study and control, each consisting of 47 participants (n = 94). Two VAG recording sessions consisted of 10 jaw open/close cycles guided by a metronome. The intraclass correlation coefficient (ICC) was calculated for seven VAG signal features. Additionally, a k-nearest-neighbors (KNN) classifier was defined and compared with a state-of-the-art method (joint vibration analysis (JVA) decision tree). ICC indicated excellent (for the integral below 300 Hz feature), good (total integral, integral above 300 Hz, and median frequency features), moderate (integral below to integral above 300 Hz ratio feature) and poor (peak amplitude feature) reliability. The accuracy scores for the KNN classifier (up to 0.81) were higher than those for the JVA decision tree (up to 0.60). The results of this study could open up a new field of research focused on the features of the vibroarthrogram in the context of the TMJ, further improving the diagnosing process.

Measuring repeatability of compositional diet estimates: An example using quantitative fatty acid signature analysis

By measuring the temporal consistency, or repeatability, in the diets of predators, we can gain a better understanding of the degree of individual specialization in resource utilization and implications for predator–prey interactions, population dynamics, and food web structure. To measure repeatability, we require repeated diet estimates of individuals over time, such as those derived from quantitative fatty acid signature analysis (QFASA), a popular diet estimation technique. However, diet estimates are often lengthy compositional vectors with many zeros, as some prey will not be consumed by all individuals, precluding the use of previously proposed measures of repeatability. In this paper, we propose a novel approach for inferring repeatability for multivariate data and, in particular, compositional diet estimates. We extend the commonly used measure of repeatability for univariate data to the multivariate compositional setting by utilizing the mean squares obtained from a nonparametric multivariate analysis of variance, and an appropriate choice of statistical distance. Our measure and its extension are compatible with both balanced and unbalanced data sets. Associated confidence intervals via nonparametric bootstrapping are also developed for the case of QFASA diet estimates that incorporate both sampling error and measurement error, where the latter error arises because the diets of predators are estimated. Simulation study results suggest that for practical levels of repeatability, our methods yield confidence intervals with the desired coverage probability even when the sample size relative to the dimension of the data (i.e., number of prey species eaten) is small. We tested our methods using QFASA diet estimates for free‐ranging Northwest Atlantic grey seals. Given the importance of understanding how predator diets vary over time and space, our method may find broad application to other compositional diet estimates, including those derived from the stomach or fecal contents, and stable isotope analyses.

Design Effect in Multilevel Settings: A Commentary on a Latent Variable Modeling Procedure for Its Evaluation

A latent variable modeling-based procedure is discussed that permits to readily point and interval estimate the design effect index in multilevel settings using widely circulated software. The method provides useful information about the relationship of important parameter standard errors when accounting for clustering effects relative to conducting single-level analyses. The approach can also be employed as an addendum to point and interval estimation of the intraclass correlation coefficient in empirical research. The discussed procedure makes it easily possible to evaluate the design effect in two-level studies by utilizing the popular latent variable modeling methodology and is illustrated with an example.

Gut microbiota individuality is contingent on temporal scale and age in wild meerkats

Inter-individual differences in gut microbiota composition are hypothesized to generate variation in host fitness-a premise for the evolution of host-gut microbe symbioses. However, recent evidence suggests that gut microbial communities are highly dynamic, challenging the notion that individuals harbour unique gut microbial phenotypes. Leveraging a long-term dataset of wild meerkats, we reconcile these concepts by demonstrating that the relative importance of identity for shaping gut microbiota phenotypes depends on the temporal scale. Across meerkat lifespan, year-to-year variation overshadowed the effects of identity and social group in predicting gut microbiota composition, with identity explaining on average less than 2% of variation. However, identity was the strongest predictor of microbial phenotypes over short sampling intervals (less than two months), predicting on average 20% of variation. The effect of identity was also dependent on meerkat age, with the gut microbiota becoming more individualized and stable as meerkats aged. Nevertheless, while the predictive power of identity was negligible after two months, gut microbiota composition remained weakly individualized compared to that of other meerkats for up to 1 year. These findings illuminate the degree to which individualized gut microbial signatures can be expected, with important implications for the time frames over which gut microbial phenotypes may mediate host physiology, behaviour and fitness in natural populations.

Videoconference-Based Physical Performance Tests: Reliability and Feasibility Study

Validated tools to evaluate physical performance remotely with real-time supervision are lacking. We assessed test−retest and inter-rater reliability, as well as the feasibility of carrying out the five-repetition sit-to-stand (5RSTS), kneeling push-up (KPU) and Shirado−Ito trunk flexor endurance (SIF) tests by 1:1 real-time videoconference. We also evaluated the correlation of these tests with measures of self-reported physical fitness, physical activity, health state and pain. A total of 96 healthy adults participated in the study (18−65 years). Relative and absolute reliabilities were assessed with the intraclass correlation coefficient (ICC) and standard error of measurement (SEM), respectively. Feasibility outcomes included testing duration, participant acceptability (1−5 Likert scale) and presence of adverse events. Self-reported measures were obtained with validated online questionnaires, and correlations were analyzed with Pearson’s partial correlation coefficients controlling for age. ICCs were excellent (>0.9), and SEMs were generally low (2.43−16.21%). The mean duration of all tests was <5 min, mean acceptability was ≥4.5, and adverse events were few. The KPU showed statistically significant correlations with various self-reported measures (p < 0.05). In conclusion, the 5RSTS, KPU and SIF were reliable and feasible when conducted by 1:1 real-time videoconference. This study provides a tool that could be logistically and economically advantageous.

Test-retest reliability of fMRI during an emotion processing task: Investigating the impact of analytical approaches on ICC values

Test-retest reliability of fMRI is often assessed using the intraclass correlation coefficient (ICC), a numerical representation of reliability. Reports of low reliability at the individual level may be attributed to analytical approaches and inherent bias/error in the measures used to calculate ICC. It is unclear whether low reliability at the individual level is related to methodological decisions or if fMRI is inherently unreliable. The purpose of this study was to investigate methodological considerations when calculating ICC to improve understanding of fMRI reliability. fMRI data were collected from adolescent females (N=23) at pre- and post-cognitive behavioral therapy. Participants completed an emotion processing task during fMRI. We calculated ICC values using contrasts and β coefficients separately from voxelwise and network (ICA) analyses of the task-based fMRI data. For both voxelwise analysis and ICA, ICC values were higher when calculated using β coefficients. This work provides support for the use of β coefficients over contrasts when assessing reliability of fMRI, and the use of contrasts may underlie low reliability estimates reported in the existing literature. Continued research in this area is warranted to establish fMRI as a reliable measure to draw conclusions and utilize fMRI in clinical settings.

Individual repeatability of avian migration phenology: a systematic review and meta-analysis

Changes in phenology and distribution are being widely reported for many migratory species in response to shifting environmental conditions. Understanding these changes and the situations in which they occur can be aided by understanding consistent individual differences in phenology and distribution and the situations in which consistency varies in strength or detectability. Studies tracking the same individuals over consecutive years are increasingly reporting migratory timings to be a repeatable trait, suggesting that flexible individual responses to environmental conditions may contribute little to population-level changes in phenology and distribution. However, how this varies across species and sexes, across the annual cycle and in relation to study (tracking method, study design) and/or ecosystem characteristics is not yet clear. Here, we take advantage of the growing number of publications in movement ecology to perform a phylogenetic multilevel meta-analysis of repeatability estimates for avian migratory timings to investigate these questions. Of 2,433 reviewed studies, 54 contained suitable information for meta-analysis, resulting in 177 effect sizes from 47 species. Individual repeatability of avian migratory timings averaged 0.414 (95% confidence interval: 0.3-0.5) across landbirds, waterbirds and seabirds, suggesting consistent individual differences in migratory timings is a common feature of migratory systems. Timing of departure from the non-breeding grounds was more repeatable than timings of arrival at or departure from breeding grounds, suggesting that conditions encountered on migratory journeys and outcome of breeding attempts can influence individual variation. Population-level shifts in phenology could arise through individual timings changing with environmental conditions and/or through shifts in the numbers of individuals with different timings. Our findings suggest that, in addition to identifying the conditions associated with individual variation in phenology, exploring the causes of between-individual variation will be key in predicting future rates and directions of changes in migratory timings. We therefore encourage researchers to report the within- and between- individual variance components underpinning the reported repeatability estimates to aid interpretation of migration behaviour. In addition, the lack of studies in the tropics means that levels of repeatability in less strongly seasonal environments are not yet clear.

Reliability of Performing Multiparametric Ultrasound in Adult Livers

PURPOSE

The aim of the study was to test inter-observer and intra-observer reliability of measuring multiparametric ultrasound in adult livers.

METHODS

We prospectively measured shear wave velocity (SWV, m/s), shear wave dispersion slope (SWD, m/s/kHz), attenuation coefficient (ATI, dB/cm/MHz), normalized local variance (NLV), and echo intensity ratio of liver to kidney (L/K ratio) in 21 adults who underwent liver magnetic resonance imaging-proton density fat fraction (MRI-PDFF). Intraclass correlation coefficient and 95% Bland-Altman limits of agreement (95% LOA) were used to analyze intra- and inter-observer reproducibility.

RESULTS

Based on liver MRI-PDFF, 21 participants (8 men and 13 women, mean age 55 years) were divided into group 1 (11 normal livers, MRI-PDFF <5%) and group 2 (10 steatotic livers, MRI-PDFF ≥5%). ICCs for intra-observer repeatability and inter-observer reproducibility in measuring multiple ultrasound parameters in both normal and steatotic livers were above 0.75. However, 95% confidence interval for measuring SWD in all livers and L/K ratio in normal livers was 0.38-0.90 and 0.47-0.91, respectively. Differences in SWV, SWD, ATI, NLV, L/K ratio, and MRI-PDFF between participants with and without hepatic steatosis were significant (p < .05), whereas serum biomarkers and body mass index were not (p > .05), based on a two-tailed t-test.

CONCLUSIONS

The results suggest that the repeatability and reproducibility for measuring liver SWV, ATI, and NLV are moderate to excellent, while those for SWD and L/K ratio are poor. Standardized machine settings, scanning protocols, and operator training are suggested in performing multiparametric ultrasound of the liver.

Reliability of task-evoked neural activation during face-emotion paradigms: Effects of scanner and psychological processes

Assessing and improving test–retest reliability is critical to efforts to address concerns about replicability of task‐based functional magnetic resonance imaging. The current study uses two statistical approaches to examine how scanner and task‐related factors influence reliability of neural response to face‐emotion viewing. Forty healthy adult participants completed two face‐emotion paradigms at up to three scanning sessions across two scanners of the same build over approximately 2 months. We examined reliability across the main task contrasts using Bayesian linear mixed‐effects models performed voxel‐wise across the brain. We also used a novel Bayesian hierarchical model across a predefined whole‐brain parcellation scheme and subcortical anatomical regions. Scanner differences accounted for minimal variance in temporal signal‐to‐noise ratio and task contrast maps. Regions activated during task at the group level showed higher reliability relative to regions not activated significantly at the group level. Greater reliability was found for contrasts involving conditions with clearly distinct visual stimuli and associated cognitive demands (e.g., face vs. nonface discrimination) compared to conditions with more similar demands (e.g., angry vs. happy face discrimination). Voxel‐wise reliability estimates tended to be higher than those based on predefined anatomical regions. This work informs attempts to improve reliability in the context of task activation patterns and specific task contrasts. Our study provides a new method to estimate reliability across a large number of regions of interest and can inform researchers' selection of task conditions and analytic contrasts.

Terminology use in animal personality research: a self-report questionnaire and a systematic review

Whether animal personality studies provide insights of broader evolutionary and ecological relevance to the field of behavioural ecology is frequently questioned. One of the sources of controversy is the vast, but often vague terminology present in the field. From a statistical perspective, animal personality is defined as among-individual variance in behaviour. However, numerous conceptual definitions of animal personality are available in the literature. Here, we performed (i) a self-report questionnaire and (ii) a systematic literature review to quantify how researchers interpreted conceptual and statistical definitions commonly used in animal personality research. We also compared whether data obtained from the questionnaire matched with data from the literature review. Among the 430 self-reported researchers that participated in our questionnaire, we observed discrepancies in key questions such as the conceptual definition of animal personality or the interpretation of repeatability. Furthermore, our literature review generally confirmed the global patterns revealed by the questionnaire. Overall, we identified common disagreements within the field of animal personality and discussed potential solutions. We advocate for adopting a terminology that avoids ambiguous interpretations and helps to make more explicit the widespread connotations implicit in the label 'animal personality'.

Rater-Reliability of Assessing Driving Errors with a DriveSafety 250 Simulator

This study aimed to establish inter-rater reliability among three raters while training new driver rehabilitation specialists to correctly identify driving errors on a DriveSafety 250 driving simulator. Five participants completed adaptation, residential and suburban, and city and highway scenarios. Intraclass correlation coefficients indicated scores between .623-.877 (p = .003-.122) for the total driving errors recorded in the two scenario drives with rater agreement initially ranging between 7-8%. When analyzing the data for types of driving errors, the intraclass correlation coefficients ranged from .556-.973 (p < .05) and rater agreement between 15-100%. Through proper training and strategy development, raters reached 100% consensus on all aspects of inter-rater reliability while assessing driving errors.

The Tongue and Mouth Imagery Questionnaire (TMIQ) for Assessing Motor Imagery vividness of the temporomandibular region: a reliability and validity case-control study

Background

To date, no validated assessment of motor imagery (MI) ability with temporomandibular disorders (TMD) exists preventing identification of good imagers and appropriate MI use during TMD rehabilitation.

Objective

To assess the reliability and construct validity of the previously developed Tongue and Mouth Imagery Questionnaire (TMIQ) compared with the gold‐standard Kinaesthetic and Visual Imagery Questionnaire (KVIQ‐10).

Methods

Both KVIQ‐10 and TMIQ assess MI ability using vividness (i.e. clarity/brightness for visual MI, _VMI; or intensity for kinesthetic MI, _KMI) of MI using a 5‐point Likert scale (1: no image/sensation, 5: clear/intense image/sensation). The KVIQ‐10 was administered once (test) and the TMIQ twice (test–retest) to heathy participants and patients with TMD. Questionnaire validity was investigated using concurrent validity (Pearson correlation and paired t test); TMIQ‐test–retest reliability (intraclass correlation coefficients, ICCs); internal consistency (Cronbach ⍺) and the factorial structure (principal factor extraction).

Results

A total of 94 participants were included (n = 47 per group). The mean vividness scores of the KVIQ‐10 and the TMIQ were significantly correlated, and not significantly different for both groups indicating concurrent validity. ICCs in the control group (range: 0.82‐0.90), and in the TMD group (range: 0.75‐0.82) indicated good reproducibility. The Cronbach ⍺ values were all above 0.94, indicating excellent reliability. Two factors were extracted corresponding to _VMI and _KMI, and explained 66% of total variance.

Conclusion

The TMIQ is a valid and reproducible MI questionnaire showing excellent internal consistency and, therefore, can be used to assess imagined movements of the TM region in healthy individuals and patients with TMD.

Evaluation of respiratory rate monitoring performance using a home oxygen monitoring device among patients with interstitial lung disease and chronic obstructive pulmonary disease

Background

Home monitoring devices have been developed to measure adherence to home oxygen therapy. In this study, we evaluated the performance of TeleOx^®, a commercially available remote monitoring device, in comparison with polysomnography (PSG) in patients with interstitial lung disease (ILD) and chronic obstructive pulmonary disease (COPD) and the factors that affected TeleOx^® correct use.

Methods

TeleOx^® was connected on the patient or concentrator side. The oxygen flow rates were set at 1, 3, and 5 L/min. Intraclass correlation coefficient (ICC) (2,1) was used to determine the agreement between respiratory rate measured using TeleOx^® and that measured using PSG, and the minimum acceptable level of reliability was >0.7.

Results

In total, 22 patients (16 with ILD and 6 with COPD) were assessed. In patients with ILD, the detection rate of patients' respiration assessed using TeleOx^® did not change according to the device's position. It increased from 53.5% to 79.0% by changing the position from the concentrator to the patient side in patients with COPD. The ICC (2,1) value indicated that TeleOx^® had acceptable reliability at oxygen flow rates of 1 and 3 L/min regardless of the device's position in patients with ILD (the concentrator side: 0.9 and 0.82, respectively; the patient side: 0.95 and 0.82, respectively), whereas that did only at the oxygen flow rate of 1 L/min and in connecting TeleOx^® on the patient side in patients with COPD (0.73).

Conclusion

The monitoring performance of TeleOx^® differed according to its position, oxygen flow rates, and patients' diseases.

Intra-examiner and inter-examiner reliability of rehabilitative ultrasound imaging for lumbar multifidus and anterolateral abdominal muscles in females with recurrent low back pain: an observational, cross-sectional study

Aim: To examine the reliability of rehabilitative ultrasound imaging performed to measure the thickness of the transverse abdominis, internal oblique, external oblique, and lumbar multifidus muscles in females with recurrent low back pain. Material and methods: A sample of 15 women was recruited. Two independent examiners recorded the thickness of their deep abdominal and spinal muscles by rehabilitative ultrasound imaging. Imaging scans of the transverse abdominis, internal oblique, and external oblique muscles were performed in the supine position and in the midaxillary line, between the lower edge of the ribcage and the iliac crest. Imaging of the lumbar multifidus was done in the prone position and at the level of the L5/S1 zygapophyseal joints. Imaging scans were performed bilaterally in rest and contraction, three times by the first examiner (at baseline, after two hours, and one week later) and once by the second examiner. Results: Good to excellent within-session intra-rater (ICC = 0.76, 0.97), good to excellent between-session intra-rater reliability (ICC = 0.73, 0.93), and good to excellent inter-rater reliability (ICC = 0.73, 0.98) were obtained. Conclusions: The results showed that rehabilitative ultrasound imaging can be used as an excellent reliable instrument by one or two examiners to measure the thickness of the transverse abdominis, internal oblique, external oblique and lumbar multifidus muscles in females with recurrent low back pain.

White Matter Functional Connectivity in Resting-State fMRI: Robustness, Reliability, and Relationships to Gray Matter

A comprehensive characterization of the spatiotemporal organization in the whole brain is critical to understand both the function and dysfunction of the human brain. Resting-state functional connectivity (FC) of gray matter (GM) has helped in uncovering the inherent baseline networks of brain. However, the white matter (WM), which composes almost half of brain, has been largely ignored in this characterization despite studies indicating that FC in WM does change during task and rest functional magnetic resonance imaging (fMRI). In this study, we identify 9 white matter functional networks (WM-FNs) and 9 gray matter functional networks (GM-FNs) of resting fMRI. Intraclass correlation coefficient (ICC) was calculated on multirun fMRI data to estimate the reliability of static functional connectivity (SFC) and dynamic functional connectivity (DFC). Associations between SFC, DFC, and their respective ICCs are estimated for GM-FNs, WM-FNs, and GM-WM-FNs. SFC of GM-FNs were stronger than that of WM-FNs, but the corresponding DFC of GM-FNs was lower, indicating that WM-FNs were more dynamic. Associations between SFC, DFC, and their ICCs were similar in both GM- and WM-FNs. These findings suggest that WM fMRI signal contains rich spatiotemporal information similar to that of GM and may hold important cues to better establish the functional organization of the whole brain.

Reliability of Aqueous Flare Measurements During Uveitis by a Spot Fluorometer

Purpose: To evaluate the reliability of intensity of light scatter (ILS) for diagnosing mild-to-moderate uveitis, which is difficult to score by the subjective standardized uveitis nomenclature (SUN) grading. Methods: ILS (in millivolts) was measured in healthy subjects and uveitis patients by a custom-made Spot fluorometer. The reliability was assessed by noise analysis of the instrument using turbid samples and intra- and interobserver variabilities. Results: ILS increased with the severity of inflammation: healthy (0.1498 ± 0.0564) and SUN0 (0.1590 ± 0.0564) < SUN1+ (0.2050 ± 0.0887) < SUN2+ (0.4283 ± 0.1895). The increase in ILS was more significant between healthy and SUN1+ (P < 0.0001) compared to the difference between SUN0 and SUN1+ (P = 0.0006). The instrument noise was small (σ = 0.002). Intraobserver measurements revealed no variability (P > 0.98) between measurements and predicted repeatability coefficients of 0.0754, 0.0522, and 0.0532 in healthy, SUN0, and SUN1+, respectively. Interobserver measurements also showed no variability (P > 0.88) and predicted intraclass correlation coefficient > 0.63 in healthy and uveitis subjects. Conclusions: The fluorometer enables precise measurement of ILS with negligible instrument noise, and the measurements are unaffected by intra- and interobserver variabilities. However, the difference in mean ILS between SUN0 and SUN1+ is greater than the instrument noise and comparable to intra- and interobserver variabilities. Thus, ILS measurements are likely impacted by blinking, eye movements, and focusing jitter. The impact of these factors must be reduced before using the fluorometer to distinguish mild-to-moderate uveitis.

Reliability of Repeated Trials Protocols for Body Composition Assessment by Air Displacement Plethysmography

Air displacement plethysmography (ADP) is fast, accurate, and reliable. Nevertheless, in about 3% of the cases, standard ADP tests provide rogue results. To spot these outliers and improve precision, repeated trials protocols have been devised, but few works have addressed their reliability. This study was conducted to evaluate the test–retest reliabilities of two known protocols and a new one, proposed here. Ninety-two healthy adults (46 men and 46 women) completed six consecutive ADP tests. To evaluate the reliability of single measurements, we used the results of the first two tests; for multiple measures protocols, we computed the test result from trials 1–3 and the retest result from trials 4–6. Bland–Altman analysis revealed that the bias and the width of the 95% interval of agreement were smaller for multiple trials than for single ones. For percent body fat (%BF)/fat-free mass, the technical error of measurement was 1% BF/0.68 kg for single trials and 0.62% BF/0.46 kg for the new protocol of multiple trials, which proved to be the most reliable. The minimal detectable change (MDC) was 2.77% BF/1.87 kg for single trials and 1.72% BF/1.26 kg for the new protocol.

A guide to the measurement and interpretation of fMRI test-retest reliability

The test-retest reliability of functional neuroimaging data has recently been a topic of much discussion. Despite early conflicting reports, converging reports now suggest that test-retest reliability is poor for standard univariate measures-namely, voxel- and region-level task-based activation and edge-level functional connectivity. To better understand the implications of these recent studies requires understanding the nuances of test-retest reliability as commonly measured by the intraclass correlation coefficient (ICC). Here we provide a guide to the measurement and interpretation of test-retest reliability in functional neuroimaging and review major findings in the literature. We highlight the importance of making choices that improve reliability so long as they do not diminish validity, pointing to the potential of multivariate approaches that improve both. Finally, we discuss the implications of recent reports of low test-retest reliability in the context of ongoing work in the field.

Validation of a Low-Cost Electrocardiography (ECG) System for Psychophysiological Research

BACKGROUND AND OBJECTIVE

The reliability of low-cost mobile systems for recording Electrocardiographic (ECG) data is mostly unknown, posing questions regarding the quality of the recorded data and the validity of the extracted physiological parameters. The present study compared the BITalino toolkit with an established medical-grade ECG system (BrainAmp-ExG).

METHODS

Participants underwent simultaneous ECG recordings with the two instruments while watching pleasant and unpleasant pictures of the "International Affective Picture System" (IAPS). Common ECG parameters were extracted and compared between the two systems. The Intraclass Correlation Coefficients (ICCs) and the Bland-Altman Limits of Agreement (LoA) method served as criteria for measurement agreement.

RESULTS

All but one parameter showed an excellent agreement (>80%) between both devices in the ICC analysis. No criteria for Bland-Altman LoA and bias were found in the literature regarding ECG parameters.

CONCLUSION

The results of the ICC and Bland-Altman methods demonstrate that the BITalino system can be considered as an equivalent recording device for stationary ECG recordings in psychophysiological experiments.

Trait Interindividual Differences in the Magnitude of Subjective Sleepiness from Sleep Inertia

In shift work settings and on-call operations, workers may be at risk of sleep inertia when called to action immediately after awakening from sleep. However, individuals may differ substantially in their susceptibility to sleep inertia. We investigated this using data from a laboratory study in which 20 healthy young adults were each exposed to 36 h of total sleep deprivation, preceded by a baseline sleep period and followed by a recovery sleep period, on three separate occasions. In the week prior to each laboratory session and on the corresponding baseline night in the laboratory, participants either extended their sleep period to 12 h/day or restricted it to 6 h/day. During periods of wakefulness in the laboratory, starting right after scheduled awakening, participants completed neurobehavioral tests every 2 h. Testing included the Karolinska Sleepiness Scale to measure subjective sleepiness, for which the data were analyzed with nonlinear mixed-effects regression to quantify sleep inertia. This revealed considerable interindividual differences in the magnitude of sleep inertia, which were highly stable within individuals after both baseline and recovery sleep periods, regardless of study condition. Our results demonstrate that interindividual differences in subjective sleepiness due to sleep inertia are substantial and constitute a trait.

Measuring Core Body Temperature Using a Non-invasive, Disposable Double-Sensor During Targeted Temperature Management in Post-cardiac Arrest Patients

Background: Precisely measuring the core body temperature during targeted temperature management after return of spontaneous circulation is mandatory, as deviations from the recommended temperature might result in side effects such as electrolyte imbalances or infections. However, previous methods are invasive and lack easy handling. A disposable, non-invasive temperature sensor using the heat flux approach (Double Sensor), was tested against the standard method: an esophagus thermometer.

Methods: The sensor was placed on the forehead of adult patients (n = 25, M/F, median age 61 years) with return of spontaneous circulation after cardiac arrest undergoing targeted temperature management. The recorded temperatures were compared to the established measurement method of an esophageal thermometer. A paired t-test was performed to examine differences between methods. A Bland-Altman-Plot and the intraclass correlation coefficient were used to assess agreement and reliability. To rule out possible influence on measurements, the patients' medication was recorded as well.

Results: Over the span of 1 year and 3 months, data from 25 patients were recorded. The t-test showed no significant difference between the two measuring methods (t = 1.47, p = 0.14, n = 1,319). Bland-Altman results showed a mean bias of 0.02°C (95% confidence interval 0.00–0.04) and 95% limits of agreement of −1.023°C and 1.066°C. The intraclass correlation coefficient was 0.94. No skin irritation or allergic reaction was observed where the sensor was placed. In six patients the bias differed noticeably from the rest of the participants, but no sex-based or ethnicity-based differences could be identified. Influences on the measurements of the Double Sensor by drugs administered could also be ruled out.

Conclusions: This study could demonstrate that measuring the core body temperature with the non-invasive, disposable sensor shows excellent reliability during targeted temperature management after survived cardiac arrest. Nonetheless, clinical research concerning the implementation of the sensor in other fields of application should be supported, as well as verifying our results by a larger patient cohort to possibly improve the limits of agreement.

Temporal variability of indoor dust concentrations of semivolatile organic compounds

The determinants of the temporal variability of indoor dust concentrations of semivolatile organic compounds (SVOCs) remain mostly unexplored. We examined temporal variability of dust concentrations and factors affecting dust concentrations for a wide range of SVOCs. We collected dust samples three times from 29 California homes during a period of 22 months and quantified concentrations of 47 SVOCs in 87 dust samples. We computed intraclass correlation coefficients (ICCs) using three samples collected within the same house. We calculated correlation coefficients (r) between two seasons with similar climate (spring and fall) and between two seasons with opposite climate (summer and winter). Among 26 compounds that were detected in more than 50% of the samples at all three visits, 20 compounds had ICCs above 0.50 and 6 compounds had ICCs below 0.50. For 19 out of 26 compounds, correlation coefficients between spring and fall (r = 0.48-0.98) were higher than those between summer and winter (r = 0.09-0.92), implying seasonal effects on dust concentrations. Our study showed that within-home temporal variability of dust concentrations was small (ICC > 0.50) for most SVOCs, but dust concentrations may vary over time for some SVOCs with seasonal variations in source rates, such as product use.

Reliability and Validity of the Polar V800 Sports Watch for Estimating Vertical Jump Height

This study aimed to assess the reliability and validity of the Polar V800 to measure vertical jump height. Twenty-two physically active healthy men (age: 22.89 ± 4.23 years; body mass: 70.74 ± 8.04 kg; height: 1.74 ± 0.76 m) were recruited for the study. The reliability was evaluated by comparing measurements acquired by the Polar V800 in two identical testing sessions one week apart. Validity was assessed by comparing measurements simultaneously obtained using a force platform (gold standard), high-speed camera and the Polar V800 during squat jump (SJ) and countermovement jump (CMJ) tests. In the test-retest reliability, high intraclass correlation coefficients (ICCs) were observed (mean: 0.90, SJ and CMJ) in the Polar V800. There was no significant systematic bias ± random errors (p > 0.05) between test-retest. Low coefficients of variation (<5%) were detected in both jumps in the Polar V800. In the validity assessment, similar jump height was detected among devices (p > 0.05). There was almost perfect agreement between the Polar V800 compared to a force platform for the SJ and CMJ tests (Mean ICCs = 0.95; no systematic bias ± random errors in SJ mean: -0.38 ± 2.10 cm, p > 0.05). Mean ICC between the Polar V800 versus high-speed camera was 0.91 for the SJ and CMJ tests, however, a significant systematic bias ± random error (0.97 ± 2.60 cm; p = 0.01) was detected in CMJ test. The Polar V800 offers valid, compared to force platform, and reliable information about vertical jump height performance in physically active healthy young men.

[Reproducibility Analysis of Iodine Concentrations of Abdominal Parenchymal Organs Based on Spectral CT]

Objective To investigate the intra-and inter-observer reproducibility of iodine concentrations of abdominal parenchymal organs based on spectral CT.Methods The water-free iodine images of the venous phase were retrospectively obtained from 50 patients with abdominal dynamic spectral CT scans.The iodine concentrations were measured in the left,right and caudate lobes of liver,spleen,pancreas and bilateral kidneys.Intraclass correlation coefficient(ICC)and Bland-Altman plot were employed to analyze the intra-and inter-observer reproducibility.Results The intra-observer ICCs of the left,right and caudate lobes of liver,spleen,pancreas,and left and right kidneys were 0.938(0.894,0.965),0.932(0.884,0.961),0.939(0.895,0.965),0.947(0.909,0.970),0.912(0.851,0.949),0.946(0.906,0.969)and 0.907(0.842,0.946),which indicated good intra-observer reproducibility.The inter-observer ICCs of the left,right and caudate lobes of liver,spleen,pancreas,and left and right kidneys were 0.947(0.909,0.970),0.927(0.875,0.958),0.943(0.902,0.968),0.956(0.924,0.975),0.934(0.887,0.962),0.927(0.875,0.958)and 0.892(0.818,0.937),which indicated good inter-observer reproducibility.Bland-Altman plots presented that more than 95% points of the intra-observer differences located within 95% CI of limits of agreement for the caudate lobe of liver,spleen,pancreas and bilateral kidneys,which was same as inter-observer differences of the caudate lobe of liver,spleen and right kidney.Conclusion The iodine concentration measurement based on the spectral CT presented good intra-and inter-observer reproducibility for the caudate lobe of liver and spleen.

Evaluation of Erythema Severity in Dermatoscopic Images of Canine Skin: Erythema Index Assessment and Image Sampling Reliability

The regular monitoring of erythema, one of the most important skin lesions in atopic (allergic) dogs, is essential for successful anti-allergic therapy. The smartphone-based dermatoscopy enables a convenient way to acquire quality images of erythematous skin. However, the image sampling to evaluate erythema severity is still done manually, introducing result variability. In this study, we investigated the correlation between the most popular erythema indices (EIs) and dermatologists’ erythema perception, and we measured intra- and inter-rater variability of the currently-used manual image-sampling methods (ISMs). We showed that the EI_BRG, based on all three RGB (red, green, and blue) channels, performed the best with an average Spearman coefficient of 0.75 and a typical absolute disagreement of less than 14% with the erythema assessed by clinicians. On the other hand, two image-sampling methods, based on either selecting specific pixels or small skin areas, performed similarly well. They achieved high intra- and inter-rater reliability with the intraclass correlation coefficient (ICC) and Krippendorff’s alpha well above 0.90. These results indicated that smartphone-based dermatoscopy could be a convenient and precise way to evaluate skin erythema severity. However, better outlined, or even automated ISMs, are likely to improve the intra- and inter-rater reliability in severe erythematous cases.

Mixed-effects models for the design and analysis of stepped wedge cluster randomized trials: An overview

The stepped wedge cluster randomized design has received increasing attention in pragmatic clinical trials and implementation science research. The key feature of the design is the unidirectional crossover of clusters from the control to intervention conditions on a staggered schedule, which induces confounding of the intervention effect by time. The stepped wedge design first appeared in the Gambia hepatitis study in the 1980s. However, the statistical model used for the design and analysis was not formally introduced until 2007 in an article by Hussey and Hughes. Since then, a variety of mixed-effects model extensions have been proposed for the design and analysis of these trials. In this article, we explore these extensions under a unified perspective. We provide a general model representation and regard various model extensions as alternative ways to characterize the secular trend, intervention effect, as well as sources of heterogeneity. We review the key model ingredients and clarify their implications for the design and analysis. The article serves as an entry point to the evolving statistical literatures on stepped wedge designs.

Image Contrast, Image Pre-Processing, and T1 Mapping Affect MRI Radiomic Feature Repeatability in Patients with Colorectal Cancer Liver Metastases

Imaging biomarkers require technical, biological, and clinical validation to be translated into robust tools in research or clinical settings. This study contributes to the technical validation of radiomic features from magnetic resonance imaging (MRI) by evaluating the repeatability of features from four MR sequences: pre-contrast T1- and T2-weighted images, pre-contrast quantitative T1 maps (qT1), and contrast-enhanced T1-weighted images. Fifty-one patients with colorectal cancer liver metastases were scanned twice, up to 7 days apart. Repeatability was quantified using the intraclass correlation coefficient (ICC) and repeatability coefficient (RC), and the impact of non-Gaussian feature distributions and image normalisation was evaluated. Most radiomic features had non-Gaussian distributions, but Box-Cox transformations enabled ICCs and RCs to be calculated appropriately for an average of 97% of features across sequences. ICCs ranged from 0.30 to 0.99, with volume and other shape features tending to be most repeatable; volume ICC > 0.98 for all sequences. 19% of features from non-normalised images exhibited significantly different ICCs in pair-wise sequence comparisons. Normalisation tended to increase ICCs for pre-contrast T1- and T2-weighted images, and decrease ICCs for qT1 maps. RCs tended to vary more between sequences than ICCs, showing that evaluations of feature performance depend on the chosen metric. This work suggests that feature-specific repeatability, from specific combinations of MR sequence and pre-processing steps, should be evaluated to select robust radiomic features as biomarkers in specific studies. In addition, as different repeatability metrics can provide different insights into a specific feature, consideration of the appropriate metric should be taken in a study-specific context.

[Reproducibility analysis of quantitative susceptibility mapping of cerebral subcortical nuclei in healthy adults]

OBJECTIVE

To investigate the intra- and inter-scanner reproducibility of quantitative susceptibility mapping (QSM) of cerebral subcortical nuclei in healthy adults.

METHODS

QSM was performed in 21 healthy adults on two different 3.0T MR scanners, and the region of interest (ROI) method was used to measure the magnetic susceptibility value of the left subcortical nuclei (the head of the caudate, putamen, globus pallidus, thalamus, substantia nigra and red nucleus). The intraclass correlation coefficient (ICC) and Bland-Altman method were used to evaluate the inter-scanner and intra-scanner reliability.

RESULTS

The ICCs of the susceptibility value ranged from 0.90 to 0.99 for all the subcortical gray nuclei except for the head of the caudate nucleus measured on the same MR scanner by the same observer. Bland-Altman analysis revealed that the points with susceptibility differences for all the subcortical gray nuclei except for substantia nigra located in the 95% CI of limits of agreement for the same MR scanner. The ICCs of the susceptibility value for the inter-scanner was 0.49 (0.08-0.75) for the head of the caudate nuleus, 0.80 (0.57-0.91) for the putamen, 0.77 (0.51-0.90) for the globus pallidus, 0.78 (0.54-0.91) for the thalamus, 0.80 (0.56-0.91) for the substantia nigra and 0.93 (0.83-0.97) for the red nucleus. The points with susceptibility difference (95.2%, 20/21) located in the 95% CI of limits of agreement for the putamen and the thalamus measured on two different MR scanners.

CONCLUSIONS

The intra-scanner reproducibility of QSM of the subcortical gray nuclei is superior to the inter-scanner reproducibility in healthy adults.

Real-time three-dimensional echocardiography for left atrial volume assessment in Thoroughbred racehorses: Observer variability and comparison with two-dimensional echocardiography

BACKGROUND

Left atrial size predicts cardiac morbidity and mortality in humans and dogs. Real-time three-dimensional echocardiography (3DE) may be reliable for assessing left atrial volume (LAV) in horses.

OBJECTIVES

To determine intra- and interobserver variability estimates of 3DE-LAV and compare it to that of 2DE-LAV estimates.

STUDY DESIGN

Method comparison.

METHODS

3DE datasets were obtained from 40 horses, then graded for quality, creating a final study population of 22 horses. The 3DE and 2DE maximum LAV (LAV_max ) and minimum LAV (LAV_min ) were measured, and left atrial emptying volume (LA EV) and left atrial ejection fraction (LA EF) were calculated, from the same 3D dataset on four occasions using (a) a semi-automatic surface recognition algorithm and (b) a modified Simpson's method of discs. 3DE LAV measurements were repeated by a second observer.

RESULTS

For 3DE, median LAV_max was 596cm³ for observer one, and 852 cm³ for observer two, LAV_min was 373 cm³ for observer one and 533 cm³ for observer two. Low intraobserver measurement variation was observed for LAV_max and LAV_min , with horse-level intraclass correlation coefficients (ICC_horse ) for both observers between 76% and 85% (horse added as random effect). The interobserver ICC was 58% for LAV_max and 50% for LAV_min on averaged measurements (with observer added as random effect), indicating consistent differences between observers. While intraobserver variation was similar for 2DE LAV_max measurements, it was greater for LAV_min (ICC_horse  = 67%). The intermethod ICC for 3DE vs 2DE was low at 14% for LAV_max and ~0% for LAV_min , indicating less-consistent differences with method.

MAIN LIMITATIONS

Small study population, low observer number, use of different imaging modalities (fundamental frequency and octave harmonics).

CONCLUSIONS

3DE assessment of LAV was reliable, suggesting suitability for longitudinal evaluation of clinical cases. Clinicians should be aware of differences in LAV measurements between observers. More defined measurement guidelines may improve repeatability.

Sample size requirements for detecting treatment effect heterogeneity in cluster randomized trials

Cluster randomized trials (CRTs) refer to experiments with randomization carried out at the cluster or the group level. While numerous statistical methods have been developed for the design and analysis of CRTs, most of the existing methods focused on testing the overall treatment effect across the population characteristics, with few discussions on the differential treatment effect among subpopulations. In addition, the sample size and power requirements for detecting differential treatment effect in CRTs remain unclear, but are helpful for studies planned with such an objective. In this article, we develop a new sample size formula for detecting treatment effect heterogeneity in two-level CRTs for continuous outcomes, continuous or binary covariates measured at cluster or individual level. We also investigate the roles of two intraclass correlation coefficients (ICCs): the adjusted ICC for the outcome of interest and the marginal ICC for the covariate of interest. We further derive a closed-form design effect formula to facilitate the application of the proposed method, and provide extensions to accommodate multiple covariates. Extensive simulations are carried out to validate the proposed formula in finite samples. We find that the empirical power agrees well with the prediction across a range of parameter constellations, when data are analyzed by a linear mixed effects model with a treatment-by-covariate interaction. Finally, we use data from the HF-ACTION study to illustrate the proposed sample size procedure for detecting heterogeneous treatment effects.

Validity and reliability of the Danish version of the pregnancy physical activity questionnaire to assess levels of physical activity during pregnancy

INTRODUCTION

Physical inactivity during pregnancy is associated with complications on the mother and child. Thus, assessments of physical activity in pregnant women is relevant. The American validated questionnaire, Pregnancy Physical Activity Questionnaire (PPAQ), measures physical activity during pregnancy, and a translated Danish version of the questionnaire is needed.

OBJECTIVES

To translate the original version of PPAQ into Danish (PPAQ-DK) and to determine face and content validity, and reliability of PPAQ-DK in pregnant Danish women.

MATERIALS AND METHODS

PPAQ was translated into Danish through a systematic process. Subsequently, face and content validity was examined on pregnant women. PPAQ-DK was revised, and the test-retest reliability of PPAQ-DK was investigated in another sample of pregnant women, where the respondents completed the questionnaire twice with one week apart, using an Intraclass Correlation Coefficient (ICC). Internal consistency reliability was determined using Cronbach´s α.

RESULTS

The face validity of the original PPAQ was achieved by interviews of 10 pregnant women (age (mean ± SD): 31.0 ± 3.0 years; 50% in second and 50% in third trimester) and minor changes in the revised version of PPAQ-DK were performed. The content validity was accepted by an expert panel. In 66 pregnant women (aged 30.2 ± 4.8 years; 16.7, 63.6, and 19.7% in first, second and third trimester, respectively), the ICC in the test-retest reliability was 0.79 (0.68-0.87) of the overall PPAQ score. The Cronbach's α in the internal consistency reliability analysis was 0.7 in the overall PPAQ score.

CONCLUSIONS

The PPAQ-DK is within acceptable reliability and is a valid and reliable tool to measure overall physical activity level of pregnant Danish women.

There is No test-retest reliability of brain activation induced by robotic passive hand movement: A functional NIRS study

INTRODUCTION

The basic paradigm of rehabilitation is based on the brain plasticity, and for promoting it, test-retest reliability (TRR) of brain activation in which certain area of the brain is repeatedly activated is required. In this study, we investigated whether the robotic passive movement has the TRR of brain activation. While active training has been shown to have TRR, but there still have been arguments over the TRR by passive movement.

METHODS

In order to test TRR, 10 repetitive sessions and various intervals (1 day, 3 days, 7 days, 23 days, 15 min, and 6 hr) were applied to five subjects, which had the same statistical power as applying two sessions to 50 subjects. In each session, three robot speeds (0.25, 0.5, and 0.75 Hz) were applied to provide passive movement using the robot. The fNIRS signal (oxy-Hb) generated in the primary sensorimotor area (SM1) was measured on a total of 29 channels. At this time, we used activation maps and intraclass correlation coefficient (ICC) values to examine the TRR and the effect of robot speeds and intervals on TRR.

RESULTS

As a result, activation maps showed prominent variation regardless of robot speeds and interval, and the ICC value (=0.002) showed no TRR of brain activation for robotic passive movement.

CONCLUSION

The brain activation induced by the robotic passive movement alone has very poor TRR, suggesting that further enhancement is required to strengthen the TRR by complementing active user engagements.

Intra-Rater Test-Retest Reliability of a Modified Child Functioning Module, Self-Report Version

Determining disability prevalence is a growing area for population statistics, especially among young adolescents. The Washington Group on Disability Statistics is one source of reporting disabilities through functional difficulties. Yet, young adolescents self-reporting through this measure is in its infancy. The purpose of this study was to carry out an intra-rater test-retest reliability study on a modified set of items for self-reporting functional difficulties. Young adolescents (N = 74; boys = 64%; age M = 13.7, SD = 1.8) with special educational needs in Finland completed a self-reported version of the Child Functioning Module in a supervised classroom. The second administration took place two weeks later. Intraclass correlation coefficient (ICC) and Kappa (k) statistics were used to test the reliability of the items, and interpretation took place through Landis and Koch, and Cohen, respectively. The majority of items had substantial or moderate agreement, although there was only fair agreement for self-care (ICC = 0.59), concentration (ICC = 0.50), and routine changes (ICC = 0.54). Kappa statistics of behavior control were interpreted to be large (k = 0.65), and seeing (k = 0.49), walking (k = 0.49), and speaking (k = 0.49) difficulties were moderate. The majority of the items in the self-reported version of the Child Functioning Module can be used in a scale format, although some caution may be required on items of self-care and concentration when used as a dichotomous variable.

One-Year Stability of Frontoparietal Cognitive Control Network Connectivity in Recent Onset Schizophrenia: A Task-Related 3T fMRI Study

Kraepelinian theory posits that schizophrenia (SZ) is a degenerative disorder that worsens throughout the lifespan. Behavioral studies of cognition have since challenged that viewpoint, particularly in the early phases of illness. Nonetheless, the extent to which cognition remains functionally stable during the early course of illness is unclear, particularly with regard to task-associated connectivity in cognition-related brain networks. In this study, we examined the 1-year stability of the frontoparietal control network during the AX-Continuous Performance Task (AX-CPT) from a new baseline sample of 153 participants scanned at 3T, of which 29 recent onset individuals with SZ and 42 healthy control (HC) participants had follow-up data available for analysis. Among individuals that had both baseline and follow-up data, reduced functional connectivity in SZ was observed between the dorsolateral prefrontal cortex (DLPFC) and superior parietal cortex (SPC) during the high control (B cue) condition. Furthermore, this deficit was stable over time, as no significant time × diagnosis interaction or effects of time were observed and intraclass correlation coefficients were greater than 0.6 in HCs and SZ. Previous 1.5T findings showing stable deficits with no evidence of degeneration in performance or DLPFC activation in an independent SZ sample were replicated. Overall, these results suggest that the neuronal circuitry supporting cognitive control is stably impaired during the early course of illness in SZ across multiple levels of analysis with no evidence of functional decline.

Dunnock social status correlates with sperm speed, but fast sperm does not always equal high fitness

Sperm competition theory predicts that males should modulate sperm investment according to their social status. Sperm speed, one proxy of sperm quality, also influences the outcome of sperm competition because fast sperm cells may fertilize eggs before slow sperm cells. We evaluated whether the social status of males predicted their sperm speed in a wild population of dunnocks (Prunella modularis). In addition to the traditional analysis of the average speed of sperm cells per sample, we also analysed subsamples of the fastest sperm cells per sample. In other words, we systematically evaluated the effects of including different numbers of the fastest sperm in our analyses, ranging from the 5-fastest sperm cells to the 100-fastest sperm cells in a sample. We further evaluated whether fitness, defined here as the number of chicks sired per male per breeding season, relates to the sperm speed in the same population. We found that males in monogamous pairings (i.e. low levels of sperm competition), produced the slowest sperm cells, whereas subordinate males in polyandrous male-male coalitions (i.e. high levels of sperm competition) produced the fastest sperm cells. This result was consistent regardless of the number of fastest sperm included in our analyses, but statistical support was conditional on the number of sperm cells included in the analysis. Interestingly, we found no significant relationship between fitness and sperm speed, which suggests that it is possible that the differential mating opportunities across social status levelled out any possible difference. Our study also suggests that it is important to identify biologically meaningful subsets of fastest sperm and cut-offs for inclusions for assessing sperm competition via sperm speed.

Validation of a Portable Force Plate for Evaluating Postural Sway

Measurements of postural sway are used to assess physiological changes due to therapy or sport training, or to describe group differences based on activity history or disease status. Portable force plates have been widely adopted for this purpose, leading us in this study to validate with linear and nonlinear metrics the posturographic data derived from both a portable plate (Natus) when compared to an in-ground plate (Bertec). Twenty participants stood on each plate for two trials each, with and without a foam perturbation and with and without eyes open on each surface. We calculated measures of path length, range, root mean squares, sample entropy, and correlation dimensions from center of pressure traces on each plate. An intraclass correlation coefficient across trials from each plate in each condition indicated satisfactory overall reliability (ICC consistency), supporting the use of either plate for postural sway research and interventions. Additionally, our results generally supported common validity (ICC absolute agreement), though, the specific degree of similarity differed for each of the tested metrics of postural sway, especially when considering whether or not data was filtered. For situations in which participants cannot visit a laboratory (e.g. performing athletes, community dwelling clinical patients, and virus risk concerns) an in-home portable force plate is a trusted and valuable data collection tool.

Toward a Dimensional Assessment of Externalizing Disorders in Children: Reliability and Validity of a Semi-Structured Parent Interview

OBJECTIVE

This study assesses the reliability and validity of the DSM-5-based, semi-structured Clinical Parent Interview for Externalizing Disorders in Children and Adolescents (ILF-EXTERNAL).

METHOD

Participant data were drawn from the ongoing ESCAschool intervention study. The ILF-EXTERNAL was evaluated in a clinical sample of 474 children and adolescents (aged 6-12 years, 92 females) with symptoms of attention-deficit/hyperactivity disorder (ADHD). To obtain interrater reliability, the one-way random-effects, absolute agreement models of the intraclass correlation (ICC) for single ICC(1,1) and average measurements ICC(1,3) were computed between the interviewers and two independent raters for 45 randomly selected interviews involving ten interviewers. Overall agreement on DSM-5 diagnoses was assessed using Fleiss' kappa. Further analyses evaluated internal consistencies, item-total correlations as well as correlations between symptom severity and the degree of functional impairment. Additionally, parents completed the German version of the Child Behavior Checklist (CBCL) and two DSM-5-based parent questionnaires for the assessment of ADHD symptoms and symptoms of disruptive behavior disorders (FBB-ADHS; FBB-SSV), which were used to evaluate convergent and divergent validity.

RESULTS

ICC coefficients demonstrated very good to excellent interrater reliability on the item and scale level of the ILF-EXTERNAL [scale level: ICC(1,1) = 0.83-0.95; ICC(1,3) = 0.94-0.98]. Overall kappa agreement on DSM-5 diagnoses was substantial to almost perfect for most disorders (0.38 ≤ κ ≤ 0.94). With some exceptions, internal consistencies (0.60 ≤ α ≤ 0.86) and item-total correlations (0.21 ≤ r it ≤ 0.71) were generally satisfactory to good. Furthermore, higher symptom severity was associated with a higher degree of functional impairment. The evaluation of convergent validity revealed positive results regarding clinical judgment and parent ratings (FBB-ADHS; FBB-SSV). Correlations between the ILF-EXTERNAL scales and the CBCL Externalizing Problems were moderate to high. Finally, the ILF-EXTERNAL scales were significantly more strongly associated with the CBCL Externalizing Problems than with the Internalizing Problems, indicating divergent validity.

CONCLUSION

In clinically referred, school-age children, the ILF-EXTERNAL demonstrates sound psychometric properties. The ILF-EXTERNAL is a promising clinical interview and contributes to high-quality diagnostics of externalizing disorders in children and adolescents.

Trends and Between-Physician Variation in Laboratory Testing: A Retrospective Longitudinal Study in General Practice

Laboratory tests are frequently ordered by general practitioners (GPs), but little is known about time trends and between-GP variation of their use. In this retrospective longitudinal study, we analyzed over six million consultations by Swiss GPs during the decade 2009–2018. For 15 commonly used test types, we defined specific laboratory testing rates (sLTR) as the percentage of consultations involving corresponding laboratory testing requests. Patient age- and sex-adjusted time trends of sLTR were modeled with mixed-effect logistic regression accounting for clustering of patients within GPs. We quantified between-GP variation by means of intraclass correlation coefficients (ICC). Nine out of the 15 laboratory test types considered showed significant temporal increases, most eminently vitamin D (ten-year odds ratio (OR) 1.88, 95% confidence interval (CI) 1.71–2.06) and glycated hemoglobin (ten-year OR 1.87, 95% CI 1.82–1.92). Test types both subject to substantial increase and high between-GP variation of sLTR were vitamin D (ICC 0.075), glycated hemoglobin (ICC 0.101), C-reactive protein (ICC 0.202), and vitamin B12 (ICC 0.166). Increasing testing frequencies and large between-GP variation of specific test type use pointed at inconsistencies of medical practice and potential overuse.

Reliability of multimodal MRI brain measures in youth at risk for mental illness

INTRODUCTION

A new generation of large-scale studies is using neuroimaging to investigate adolescent brain development across health and disease. However, imaging artifacts such as head motion remain a challenge and may be exacerbated in pediatric clinical samples. In this study, we assessed the scan-rescan reliability of multimodal MRI in a sample of youth enriched for risk of mental illness.

METHODS

We obtained repeated MRI scans, an average of 2.7 ± 1.4 weeks apart, from 50 youth (mean age 14.7 years, SD = 4.4). Half of the sample (52%) had a diagnosis of an anxiety disorder; 22% had attention-deficit/hyperactivity disorder (ADHD). We quantified reliability with the test-retest intraclass correlation coefficient (ICC).

RESULTS

Gray matter measurements were highly reliable with mean ICCs as follows: cortical volume (ICC = 0.90), cortical surface area (ICC = 0.89), cortical thickness (ICC = 0.82), and local gyrification index (ICC = 0.85). White matter volume reliability was excellent (ICC = 0.98). Diffusion tensor imaging (DTI) components were also highly reliable. Fractional anisotropy was most consistently measured (ICC = 0.88), followed by radial diffusivity (ICC = 0.84), mean diffusivity (ICC = 0.81), and axial diffusivity (ICC = 0.78). We also observed regional variability in reconstruction, with some brain structures less reliably reconstructed than others.

CONCLUSIONS

Overall, we showed that developmental MRI measures are highly reliable, even in youth at risk for mental illness and those already affected by anxiety and neurodevelopmental disorders. Yet, caution is warranted if patterns of results cluster within regions of lower reliability.