Intra-individual Comparisons of the Ultrasound-Guided Attenuation Parameter and the Magnetic Resonance Imaging-Based Proton Density Fat Fraction Using Bias and Precision Statistics

Ultrasound-based techniques using the attenuation coefficient, including the ultrasound-guided attenuation parameter (UGAP), have been developed for the quantification of hepatic steatosis. The magnetic resonance imaging-based proton density fat fraction (MRI-PDFF) is considered to be more accurate than liver biopsy for liver fat quantification. The aim of this study was to perform intra-individual comparisons of UGAP and MRI-PDFF for determining hepatic steatosis grade. The study enrolled 309 patients who underwent UGAP and MRI-PDFF measurements. Bland-Altman analysis was conducted after transforming MRI-PDFF values to a normal distribution and converted to a common set of units using linear regression analysis for differing scales. The expected limits of agreement (LOA) was defined as the square root of the sum of the squares of UGAP and MRI-PDFF precision. A Bland-Altman plot revealed that the bias and upper and lower LOAs (ULOA and LLOA) were -0.0047, 0.1160 and -0.1255, respectively. The percentage difference indicated that the mean, ULOA and LLOA were -1.1434%, 18.1723% and -20.4590%, respectively. The calculated expected LOA was 18.5449%, and 283 of 309 patients (91.6%) had a percentage difference within 18.5449%. Bland-Altman analysis revealed that UGAP and MRI-PDFF were interchangeable within a clinically acceptable range.

The challenge of measuring spinopelvic parameters: inter-rater reliability before and after minimally invasive lumbar spondylodesis

BACKGROUND

The common manual measurement technique of spinal sagittal alignment on X-rays is susceptible to rater-dependent variability, which has not been adequately considered in previous publications. This study investigates the effect of those variations in the characterization of patients receiving lumbar spondylodesis.

METHODS

General alignment parameters on pre- and postoperative X-rays were evaluated by four raters in 43 prospectively sampled patients undergoing monolevel spondylodesis. The Intra-class Correlation Coefficient (ICC) for each rater pair and all raters together was calculated for inter-rater reliability. For the operation-induced change of the sagittal alignment in every patient the Wilcoxon test was applied to compare for each rater separately.

RESULTS

The ICCs were "good" (>0.75) to "excellent" (>0.9) for all raters together and for 45 of the 48 single rater pairs (93.75%). All revealed a significant increase of the addressed segmental lordosis and disc height and no significant change for spinopelvic parameters and sagittal vertical axis from pre- to postoperative. The lumbar lordosis showed a significant increase through the operation of +2.5° (p = 0.014) and +3.7° (p = 0.015) in two raters and no difference for the other ones (+2.1°, p = 0.171; -2.2°, p = 0.522).

CONCLUSIONS

The pre- to postoperative change of lumbar lordosis revealed different significance levels for different raters, although the ICCs were formally good. Accordingly, the evaluation by only one rater would lead to different conclusions. Due to this susceptibility of alignment measurements to rater-dependent variability, the exact evaluation process should be described in every publication and the consistency of significant results be validated through multiple raters.

TRIALS REGISTRATION

The trial was approved by the local ethics committee and listed at the national clinical trials register ( DRKS00004514 , date of registration: 08/11/2012).

Reliability of the freehand region-of-interest method in quantitative cerebral diffusion tensor imaging

Background

Diffusion tensor imaging (DTI) is a magnetic resonance imaging (MRI) technique used for evaluating changes in the white matter in brain parenchyma. The reliability of quantitative DTI analysis is influenced by several factors, such as the imaging protocol, pre-processing and post-processing methods, and selected diffusion parameters. The region-of-interest (ROI) method is most widely used of the post-processing methods because it is found in commercial software. The focus of our research was to study the reliability of the freehand ROI method using various intra- and inter-observer analyses.

Methods

This study included 40 neurologically healthy participants who underwent diffusion MRI of the brain with a 3 T scanner. The measurements were performed at nine different anatomical locations using a freehand ROI method. The data extracted from the ROIs included the regional mean values, intra- and inter-observer variability and reliability. The used DTI parameters were fractional anisotropy (FA), the apparent diffusion coefficient (ADC), and axial (AD) and radial (RD) diffusivity.

Results

The average intra-class correlation coefficient (ICC) of the intra-observer was found to be 0.9 (excellent). The single ICC results were excellent (> 0.8) or adequate (> 0.69) in eight out of the nine regions in terms of FA and ADC. The most reliable results were found in the frontobasal regions. Significant differences between age groups were also found in the frontobasal regions. Specifically, the FA and AD values were significantly higher and the RD values lower in the youngest age group (18–30 years) compared to the other age groups.

Conclusions

The quantitative freehand ROI method can be considered highly reliable for the average ICC and mostly adequate for the single ICC. The freehand method is suitable for research work with a well-experienced observer. Measurements should be performed at least twice in the same region to ensure that the results are sufficiently reliable. In our study, reliability was slightly undermined by artifacts in some regions such as the cerebral peduncle and centrum semiovale. From a clinical point of view, the results are most reliable in adults under the age of 30, when age-related changes in brain white matter have not yet occurred.

Linked Color Imaging and the Kyoto Classification of Gastritis: Evaluation of Visibility and Inter-Rater Reliability

BACKGROUND/AIMS

To compare white light imaging (WLI) with linked color imaging (LCI) and blue LASER imaging (BLI) in endoscopic findings of Helicobacter pylori presently infected, previously infected, and uninfected gastric mucosae for visibility and inter-rater reliability.

METHODS

WLI, LCI and BLI bright mode (BLI-bright) were used to obtain 1,092 endoscopic images from 261 patients according to the Kyoto Classification of Gastritis. Images were evaluated retrospectively by 10 experts and 10 trainee endoscopists and included diffuse redness, spotty redness, map-like redness, patchy redness, red streaks, intestinal metaplasia, and an atrophic border (52 cases for each finding, respectively). Physicians assessed visibility as follows: 5 (improved), 4 (somewhat improved), 3 (equivalent), 2 (somewhat decreased), and 1 (decreased). Visibility was assessed from totaled scores. The inter-rater reliability (intraclass correlation coefficient) was also evaluated.

RESULTS

Compared with WLI, all endoscopists reported improved visibility with LCI: 55.8% for diffuse redness; LCI: 38.5% for spotty redness; LCI: 57.7% for map-like redness; LCI: 40.4% for patchy redness; LCI: 53.8% for red streaks; LCI: 42.3% and BLI-bright: 80.8% for intestinal metaplasia; LCI: 46.2% for an atrophic border. For all endoscopists, the inter-rater reliabilities of LCI compared to WLI were 0.73-0.87.

CONCLUSION

The visibility of each endoscopic finding was improved by LCI while that of intestinal metaplasia was improved by BLI-bright.

Short-term variability of bisphenols in spot, morning void and 24-hour urine samples

Due to worldwide regulations on the application of the high production volume industrial chemical bisphenol A (BPA) in various consumer products, alternative bisphenols such as bisphenol F (BPF) and bisphenol S (BPS) are increasingly used. To assess human exposure to these chemicals, biomonitoring of urinary concentrations is frequently used. However, the short-term variability of alternative bisphenols has not been evaluated thoroughly yet, which is essential to achieve a correct estimation of exposure. In this study, we collected all spot urine samples from ten healthy adults for five consecutive days, and an additional 24 h pooled sample. We measured the concentrations of seven bisphenols (BPAF, BPF, BPA, BPB, BPZ, BPS and BPAP) in these samples using gas chromatography coupled to tandem mass spectrometry. BPA, BPF and BPS were frequently found in spot samples (>80%), while bisphenol AP (BPAP) was detected in 43% of spot samples. Calculations of intra-class correlation coefficients (ICCs) showed that reproducibility of these four bisphenols was relatively poor (<0.01-0.200) but improved when concentrations were corrected for urine dilution using creatinine levels (0.128-0.401). Of these four bisphenols, BPF showed the best reproducibility (ICC 0.200-0.439) and BPS the most variability (ICC <0.01-0.128). In general, the within-participant variability of bisphenol levels was the largest contributor to the total variance (47-100%). We compared repeated first morning voids to 24 h pooled urine and found no significantly different concentrations for BPA, BPF, BPS, or BPAP. Levels of BPA and BPF differed significantly depending on the sampling time throughout the day. The findings in this study suggest that collecting multiple samples per participant over a few days, in predefined time windows throughout the day, could result in a more reliable estimation of internal exposure to bisphenols.

Reliability and Agreement of Point and 2-D Shear-Wave Elastography in Assessing the Sciatic Nerve Stiffness

Information is lacking about the reliability and agreement of different shear-wave elastographic modes in the peripheral nervous system evaluation. The aim of this observational study was to evaluate reproducibility and agreement of two different shear-wave elastographic modes for measuring the sciatic nerve stiffness in patients affected by osteoarthrosis. Two sets of three measurements were conducted bilaterally on the sciatic nerve of 20 patients with point and 2-D shear-wave elastography by a unique expert sonographer. This consecutive case series study was performed in 1 mo. No significant difference was found comparing the first with the second set of evaluations (p = 0.08 for point shear-wave elastography and 0.3 for 2-D shear-wave elastography). Correlation between the two sets of measurements was good and excellent (0.799 with point shear-wave elastography and 0.877 with 2-D shear-wave elastography). Intra-class coefficient correlation between the two sets of measurements was excellent for both shear-wave elastographic modes (0.869 and 0.938, respectively); no agreement between them was demonstrated (analysis of variance [ANOVA] test: p = 0.014).Despite the lack of agreement owing to the different procedures for measuring, both shear-wave elastographic modes allow reliable stiffness measurements of the sciatic nerve and may be used to evaluate stiffness changes.

Reproducibility of manual choroidal thickness measurements using optical coherence tomography

PURPOSE

Spectral-domain optical coherence tomography (SD-OCT) is the most useful tool to measure choroidal thickness (CT). CT may be increased in ocular and systemic diseases. However, there are concerns relating reproducibility and external validity of OCT. The aim of this study was to assess the inter-observer and intra-observer variability of manual OCT measurements.

METHODS

CT was manually measured in the central choroid of 40 eyes from 21 subjects (11 healthy and 10 with ankylosing spondylitis) using RTVue-100 OCT (Optovue Inc., Fremont, CA, EE.UU.). Measurements were performed by 9 independent ophthalmologists from 6 different centers. To assess the inter-observer variability, the intra-class correlation coefficient (ICC) method was calculated. Also, intra-observer variability was assessed in 2 of the ophthalmologists.

RESULTS

The mean subfoveal CT was 364.9±85.1μm (range, 170 to 572). The inter-observer ICC was 0.823 (CI 95%, 0.749 to 0.888, p<0.001). The intra-observer ICCs were 0.885 (CI 95%, 0.783 to 0.939, p<0.001) and 0.925 (CI 95%, 0.859 to 0.960. p<0.001).

CONCLUSIONS

In this study, manual measurements of CT with OCT showed a good concordance. These results suggest that manual OCT is a valid tool for multicenter studies.

Assessment of Stability and Discrimination Capacity of Radiomic Features on Apparent Diffusion Coefficient Images

The objectives of the study are to develop a new way to assess stability and discrimination capacity of radiomic features without the need of test-retest or multiple delineations and to use information obtained to perform a preliminary feature selection. Apparent diffusion coefficient (ADC) maps were computed from diffusion-weighted magnetic resonance images (DW-MRI) of two groups of patients: 18 with soft tissue sarcomas (STS) and 18 with oropharyngeal cancers (OPC). Sixty-nine radiomic features were computed, using three different histogram discretizations (16, 32, and 64 bins). Geometrical transformations (translations) of increasing entity were applied to the regions of interest (ROIs), and the intra-class correlation coefficient (ICC) was used to compare the features computed on the original and modified ROIs. The distribution of ICC values for minimal and maximal entity translations (ICC₁₀ and ICC₁₀₀, respectively) was used to adjust thresholds of ICC (ICC_min and ICC_max) used to discriminate between good, unstable (ICC₁₀ < ICC_min), and non-discriminative features (ICC₁₀₀ > ICC_max). Fifty-four and 59 radiomic features passed the stability-based selection for all the three histogram discretizations for the OPC and STS datasets, respectively. The excluded features were similar across the different histogram discretizations (Jaccard’s index 0.77 ± 0.13 and 0.9 ± 0.1 for OPC and STS, respectively) but different between datasets (Jaccard’s index 0.19 ± 0.02). The results suggest that the observed radiomic features are mainly stable and discriminative, but the stability depends on the region of the body under observation. The method provides a way to assess stability without the need of test-retest or multiple delineations.

Maximal contraction methods influence the magnitude and reliability of global electromyographic signal characteristics

OBJECTIVE

The purpose was two-fold: (1) to examine differences in maximal voluntary isometric torque (MVIT) production, and electromyographic signal amplitude (EMG_AMP) and mean power frequency (EMG_MPF) values obtained during traditional (MVIC_TRAD), rapid (MVIC_RAPID), and ramp (MVIC_RAMP) maximal voluntary isometric contractions, and (2) to determine if there were differences in the reliability of MVIT, EMG_AMP and EMG_MPF among the three MVIC types.

APPROACH

Twenty-two young males and females completed MVIC_TRAD, MVIC_RAPID, and MVIC_RAMP muscle actions on two separate visits separated by 48 h. During all MVICs, MVIT and EMG_AMP and EMG_MPF of the vastus lateralis (VL) and rectus femoris (RF) were quantified.

MAIN RESULTS

MVIT was greater during MVIC_TRAD and MVIC_RAPID than during MVIT_RAMP (both p < 0.001). VL and RF EMG_AMP were greater during MVIC_RAMP than during MVIC_RAPID (p = 0.02 and 0.004). For EMG_MPF, there were no significant differences among MVIC types. Although all MVIC types generally resulted in reliable measurements of MVIT and EMG_AMP, reliability was stronger for EMG_MPF quantified during the MVIC_RAMP.

SIGNIFICANCE

Investigators may choose MVIC type based on preference or equipment availability. However, investigators should note that MVIC_RAMP contractions will likely yield the greatest EMG_AMP values and more reliable measurements of VL and RF EMG_MPF.

SANRA-a scale for the quality assessment of narrative review articles

Background

Narrative reviews are the commonest type of articles in the medical literature. However, unlike systematic reviews and randomized controlled trials (RCT) articles, for which formal instruments exist to evaluate quality, there is currently no instrument available to assess the quality of narrative reviews. In response to this gap, we developed SANRA, the Scale for the Assessment of Narrative Review Articles.

Methods

A team of three experienced journal editors modified or deleted items in an earlier SANRA version based on face validity, item-total correlations, and reliability scores from previous tests. We deleted an item which addressed a manuscript’s writing and accessibility due to poor inter-rater reliability. The six items which form the revised scale are rated from 0 (low standard) to 2 (high standard) and cover the following topics: explanation of (1) the importance and (2) the aims of the review, (3) literature search and (4) referencing and presentation of (5) evidence level and (6) relevant endpoint data. For all items, we developed anchor definitions and examples to guide users in filling out the form. The revised scale was tested by the same editors (blinded to each other’s ratings) in a group of 30 consecutive non-systematic review manuscripts submitted to a general medical journal.

Results

Raters confirmed that completing the scale is feasible in everyday editorial work. The mean sum score across all 30 manuscripts was 6.0 out of 12 possible points (SD 2.6, range 1–12). Corrected item-total correlations ranged from 0.33 (item 3) to 0.58 (item 6), and Cronbach’s alpha was 0.68 (internal consistency). The intra-class correlation coefficient (average measure) was 0.77 [95% CI 0.57, 0.88] (inter-rater reliability). Raters often disagreed on items 1 and 4.

Conclusions

SANRA’s feasibility, inter-rater reliability, homogeneity of items, and internal consistency are sufficient for a scale of six items. Further field testing, particularly of validity, is desirable. We recommend rater training based on the “explanations and instructions” document provided with SANRA. In editorial decision-making, SANRA may complement journal-specific evaluation of manuscripts—pertaining to, e.g., audience, originality or difficulty—and may contribute to improving the standard of non-systematic reviews.

Impact of exposure to phenols during early pregnancy on birth weight in two Canadian cohort studies subject to measurement errors

BACKGROUND

It is of interest to know whether early pregnancy exposure to phenols such as bisphenol-A (BPA) or triclosan (TCS) negatively impacts birth weight outcomes. Exposure to these chemicals is widespread in the Canadian population but obtaining accurate measurements of average exposure is difficult because these chemicals are rapidly excreted from the body, causing body levels to fluctuate both within and between days, as observed in a recent Canadian study (P4). This measurement error can attenuate the estimated effects of exposures.

METHODS

Data from two Canadian cohort studies, the Plastics and Personal-care Products use in Pregnancy (P4) Study and the Maternal-Infant Research on Environmental Chemicals (MIREC) Study, such that all participants with complete BPA or TCS exposure and outcome data were used (MIREC n = 1822, P4 n = 68). We used regression calibration to correct for the attenuating effects of exposure measurement error when modeling the effect of first trimester BPA or TCS exposure on four birth weight outcomes: birth weight (BW), low birth weight (LBW), small for gestational age (SGA) and large for gestational age (LGA). Specific gravity, time of day, and time since last urine void were also controlled in the analysis.

RESULTS

TCS exposure has a marginally significant association with SGA only with odds ratio 0.87 and 95% confidence interval (0.74, 1.00). It also has a marginally significant association with LGA in male offspring with odds ratio 1.11 and 95% confidence interval (1.00, 1.25). The effects of BPA on the four birth outcomes were insignificant.

CONCLUSIONS

Increased TCS exposure during pregnancy is marginally associated with decreased odds of having SGA offspring. It is possibly associated with decreased BW in males and decreased odds of LBW, though these associations were not present in measurement error corrected models. TCS is possibly associated with increased odds in male offspring of being LGA, though this relationship was not present in models not corrected for measurement error. The study finds no significant effects of BPA on birth weight outcomes, which may be due to more severe measurement error in a single observation of BPA.

Improved Visibility of Barrett's Esophagus with Linked Color Imaging: Inter- and Intra-Rater Reliability and Quantitative Analysis

BACKGROUND/AIMS

To evaluate the usefulness of linked color imaging (LCI) and blue LASER imaging (BLI) in Barrett's esophagus (BE) compared with white light imaging (WLI).

METHODS

Five expert and trainee endoscopists compared WLI, LCI, and BLI images obtained from 63 patients with short-segment BE. Physicians assessed visibility as follows: 5 (improved), 4 (somewhat improved), 3 (equivalent), 2 (somewhat decreased), and one (decreased). Scores were evaluated to assess visibility. The inter- and intra-rater reliability (intra-class correlation coefficient) of image assessments were also evaluated. Images were objectively evaluated based on L* a* b* color values and color differences (ΔE*) in a CIELAB color space system.

RESULTS

Improved visibility compared with WLI was achieved for LCI: 44.4%, BLI: 0% for all endoscopists; LCI: 55.6%, BLI: 1.6% for trainees; and LCI: 47.6%, BLI: 0% for experts. The visibility score of trainees compared with experts was significantly higher for LCI (p = 0.02). Intra- and inter-rater reliability ratings for LCI compared with WLI were "moderate" for trainees, and "moderate-substantial" for experts. The ΔE* revealed statistically significant differences between WLI and LCI.

CONCLUSION

LCI improved the visibility of short-segment BE compared with WLI, especially for trainees, when evaluated both subjectively and objectively.

School and class-level variations and patterns of physical activity: a multilevel analysis of Danish high school students

Background

There is limited knowledge of physical activity (PA) patterns among high school students. High schools plays an important role as context for the students, but it is uncertain to what extent schools influence student participation in PA during leisure time. The purpose of this study is to describe patterns of PA and assess variations between schools and classes in PA, in a large cohort of Danish high school students.

Methods

Self-reported cross-sectional data came from The Danish National Youth Study, comprising a total of 70,674 students attending 119 different schools and 3213 classes. Multilevel logistic regressions were applied to evaluate the association between socio-demographic variables and patterns of PA, and to assess the impact of schools and classes on PA measures.

Results

Students whose parents have achieved a lower level of education, older students and girls of perceived ethnic minority generally participated less in several forms of PA during leisure time. Substantial variations between schools were observed in terms of participation in PA at school during leisure time and in terms of use of active transportation to and from school. The school-level accounted for 9% (intra-class correlation coefficient (ICC) = 0.09 (95% CI: 0.06–0.11)) and 8% (ICC = 0.08 (95% CI: 0.07–0.11)) of the variation for participation in PA during leisure time and active transportation.

Conclusion

Overall, students whose parents achieved a lower level of education, older students and girls of perceived ethnic minority represent vulnerable groups in relation to participation in several forms of PA during leisure time. The ICCs indicate that schools, in particular, have the potential to influence participation in PA at school during leisure time and active transportation to and from school. Thus, high schools should encourage and facilitate activities aimed at engaging students in PA during leisure time as well as encourage active transportation.

Shear wave velocity measurement of upper trapezius muscle by color Doppler shear wave imaging

PURPOSE

Skeletal muscle stiffness is thought to be the result of increased tissue hardness, but measurement accuracy has been dependent on operator technique. We have proposed a novel shear wave real-time imaging method (color Doppler shear wave imaging: CD SWI) with continuous shear waves excited from the tissue surface by a mechanical vibrator.

METHODS

Using the method, shear wave velocity was measured for the upper trapezius muscle. Adaptive shear wave velocity measurement by means of quality estimation of shear wave wavefront was adopted. We recruited 23 male volunteers with no history of orthopedic disease and recorded shear wave propagation to assess the intra- and inter-observer reliability. For intra-observer reliability, one observer took two measurements separated by a time delay, and the intra-class correlation coefficient (ICC) was calculated (1,1). For inter-observer reliability, ICC (2,1) was calculated from both observers' measurements.

RESULTS

Mean propagation speed was 3.75 ± 0.47 (first) and 3.71 ± 0.49 m/s (second) for Observer A (ICC (1,1) = 0.91 [95% CI 0.76-0.96]) and 3.80 ± 0.53 m/s for Observer B (ICC (2,1) = 0.83 [95% CI 0.56-0.94]).

CONCLUSIONS

This result suggests that our technique is satisfactorily reliable and has potential for future application in various fields, such as evaluation of muscle condition or the effects of rehabilitation.

Longitudinal changes in pubertal maturation and white matter microstructure

Emerging evidence in the field of adolescent neurodevelopment suggests that pubertal processes may contribute to known trajectories of brain maturation, and may contribute, in part, to sex differences in related cognitive, behavioral and mental health outcomes. The current longitudinal study examined how changes in physical pubertal maturation (measured by the Peterson Developmental Scale) predict changes in white matter microstructure in 18 boys and 15 girls over an approximate 2-year follow-up period, while accounting for age. Using Tract-Based Spatial Statistics and multi-level modeling, the results showed that physical pubertal changes predict patterns of changes in fractional anisotropy (FA) in white matter regions in the thalamus, precentral gyrus, superior corona radiata, corpus callosum (genu), superior corona radiata, and superior frontal gyrus. Sex specific changes were also seen, as changes in gonadal and adrenal development related to increases in FA in the superior frontal gyrus and precentral gyrus in boys, but gonadal development related to decreases in FA in the anterior corona radiata in girls. These findings are the first to show how changes over time in pubertal development influence white matter development. In addition, they support a larger body of emerging research suggesting that pubertal processes contribute to distinct changes in boys and girls across brain development.

Musculoskeletal diagnostic ultrasound imaging for thickness measurement of four principal muscles of the cervical spine -a reliability and agreement study

BACKGROUND

The reliability of musculoskeletal diagnostic ultrasound imaging (MSK-DUSI) for the evaluation of neck musculature has been sparsely documented in the research literature. Until now, research has featured a limited number of subjects and only few studies have tested for both inter- and intra-reliability using appropriate methodology.

METHODS

Four examiners conducted an inter- and intra-rater reliability and agreement study. Fifty females with and without neck pain (NP) between the ages of 20-70 were recruited from October 2014 to April 2015. The muscles that were evaluated were the longus colli (Lcol), the rectus capitis posterior major (Rcpm), the deep cervical extensors (Dce) and the semispinalis capitis (Sscap). Each of the examiners captured ultrasound images of their allocated muscle and measured the thickness of that muscle twice, on separate occasions, for the first part of the intra-rater reliability study. For the second part, a second image of the same muscle was taken on the same subject and measured by the same examiner. The four examiners then met to measure on each other's images, to test inter-rater reliability. Their results were compared pair-wise using Interclass Correlation Coefficients (ICC) and Bland-Altman plots. Linear regression analysis was performed to evaluate for possible bias.

RESULTS

Inter-rater reliability was found to be good for Lcol and Sscap muscles and moderate towards poor for the deeper Rcpm and Dce muscles. Intra-rater reliability was good for all the muscles, with the exception of the Dce, which was found to be moderate in the second part of the study. The B&A plots showed good agreement, few outliers, and no bias. However, the agreement intervals indicated a measurement error within the variance of the method that may not have been acceptable for these small muscles if the aim is to evaluate change in thickness.

CONCLUSIONS

This study found that MSK-DUSI had variable reliability in assessing the thickness of the Lcol, Rcpm, Dce, and Sscap muscles. No bias was demonstrated, but agreement intervals were wide.

The relative and absolute reliability of the Functional Independence and Difficulty Scale in community-dwelling frail elderly Japanese people using long-term care insurance services

BACKGROUND

The newly developed Functional Independence and Difficulty Scale is a tool for assessing the performance of basic activities of daily living in terms of both independence and difficulty. The reliability of this new scale has not been assessed.

AIMS

The aim of this study was to examine the relative reliability and absolute reliability of the newly developed scale in community-dwelling frail elderly people in Japan.

METHODS

Participants were 47 community-dwelling elderly subjects (22 for assessing test-retest reliability and 25 for assessing inter-rater reliability). As relative reliability indices, intra-class correlation coefficients were used. From an absolute reliability perspective, we conducted Bland-Altman analysis and calculated the limit of agreement or minimal detectable change to determine the acceptable range of error.

RESULTS

Intra-class correlation coefficients for test-retest and inter-rater reliability were 0.90 (P < 0.001) and 0.97 (P < 0.001), respectively. The limit of agreement for test-retest reliability was -5.2 to 1.8, representing an increase of over six points for improvement and a decrease of over two points for decline of basic activities of daily living ability. The minimal detectable change for inter-rater reliability was 3.7, indicating that a three-point difference might be existed between difference raters. The results of this study demonstrated that the FIDS appeared to be a reliable instrument for use in Japanese community-dwelling frail elderly people.

CONCLUSIONS

While further research using a large and more diverse sample of participants is needed, our findings support the use of FIDS in clinical practice or clinical research targeting frail elderly Japanese people.

3-D Ultrasound Segmentation of the Placenta Using the Random Walker Algorithm: Reliability and Agreement

Volumetric segmentation of the placenta using 3-D ultrasound is currently performed clinically to investigate correlation between organ volume and fetal outcome or pathology. Previously, interpolative or semi-automatic contour-based methodologies were used to provide volumetric results. We describe the validation of an original random walker (RW)-based algorithm against manual segmentation and an existing semi-automated method, virtual organ computer-aided analysis (VOCAL), using initialization time, inter- and intra-observer variability of volumetric measurements and quantification accuracy (with respect to manual segmentation) as metrics of success. Both semi-automatic methods require initialization. Therefore, the first experiment compared initialization times. Initialization was timed by one observer using 20 subjects. This revealed significant differences (p < 0.001) in time taken to initialize the VOCAL method compared with the RW method. In the second experiment, 10 subjects were used to analyze intra-/inter-observer variability between two observers. Bland-Altman plots were used to analyze variability combined with intra- and inter-observer variability measured by intra-class correlation coefficients, which were reported for all three methods. Intra-class correlation coefficient values for intra-observer variability were higher for the RW method than for VOCAL, and both were similar to manual segmentation. Inter-observer variability was 0.94 (0.88, 0.97), 0.91 (0.81, 0.95) and 0.80 (0.61, 0.90) for manual, RW and VOCAL, respectively. Finally, a third observer with no prior ultrasound experience was introduced and volumetric differences from manual segmentation were reported. Dice similarity coefficients for observers 1, 2 and 3 were respectively 0.84 ± 0.12, 0.94 ± 0.08 and 0.84 ± 0.11, and the mean was 0.87 ± 0.13. The RW algorithm was found to provide results concordant with those for manual segmentation and to outperform VOCAL in aspects of observer reliability. The training of an additional untrained observer was investigated, and results revealed that with the appropriate initialization protocol, results for observers with varying levels of experience were concordant. We found that with appropriate training, the RW method can be used for fast, repeatable 3-D measurement of placental volume.

Reproducibility of two 3-D ultrasound carotid plaque quantification methods

Compared with single 2-D images, emerging 3-D ultrasound technologies hold the promise of reducing variability and increasing sensitivity in the quantification of carotid plaques for individual cardiovascular risk stratification. Inter- and intra-observer agreement between a manual, cross-sectional, 2-D freehand sweep and a mechanical 3-D ultrasound investigation of 62 carotid artery plaques is reported with intra-class correlation coefficients (with 95% confidence intervals). Inter-observer agreement was 0.60 (0.29-0.77) for the freehand method and 0.89 (0.83-0.93) for the mechanical 3-D acquisition. The use of semi-automated computerized planimetric measurements of plaque burden has high intra-observer repeatability, but is vulnerable to systematic inter-observer differences. For the 2-D freehand sweep, a considerable contribution to variation is introduced by the scanning procedure itself, that is, the lack of controlled motion along the third dimension. Future implementation of 3-D ultrasound quantification in large-scale studies of inter-individual cardiovascular risk assessment seems justified using the methods described.

Test-retest reproducibility of cannabinoid-receptor type 1 availability quantified with the PET ligand [¹¹C]MePPEP

Background

Endocannabinoids are involved in normal cognition, and dysfunction in cannabinoid-receptor-mediated neurotransmission has been suggested in a variety of neurological and psychiatric pathologies. The type 1 cannabinoid receptor (CB₁) is widely expressed in the human central nervous system. The objective of this study was to quantify the test–retest reproducibility of measures of the PET ligand [¹¹C]MePPEP in order to assess the stability of CB₁-receptor quantification in humans in vivo.

Methods

Fifteen healthy subjects (eight females; median age 32 years, range 25 to 65 years) had a 90-minute PET scan on two occasions after injection of a median dose of [¹¹C]MePPEP of 364 MBq. Metabolite-corrected arterial plasma input functions were obtained for all scans. Eight ROIs, reflecting different levels of receptor densities/concentrations, were defined automatically: hippocampus, anterior cingulate gyrus, inferior frontal gyrus, caudate nucleus, globus pallidus, nucleus accumbens, thalamus, and pons. We used seven quantification methods: reversible compartmental models with one and two tissue classes, two and four rate constants, and a variable blood volume term (2kbv; 4kbv); model-free (spectral) analyses with and without regularisation, including one with voxel-wise quantification; the simplified reference tissue model (SRTM) with pons as a pseudo-reference region; and modified standard uptake values (mSUVs) calculated for the period of ~ 30–60 min after injection. Percentage test–retest change and between-subject variability were both assessed, and test–retest reliability was quantified by the intraclass correlation coefficient (ICC). The ratio of binding estimates pallidum:pons served as an indicator of a method's ability to reflect binding heterogeneity.

Results

Neither the SRTM nor the 4kbv model produced reliable measures, with ICCs around zero. Very good (> 0.75) or excellent (> 0.80) ICCs were obtained with the other methods. The most reliable were spectral analysis parametric maps (average across regions ± standard deviation 0.83 ± 0.03), rank shaping regularised spectral analysis (0.82 ± 0.05), and the 2kbv model (0.82 ± 0.09), but mSUVs were also reliable for most regions (0.79 ± 0.13). Mean test–retest changes among the five well-performing methods ranged from 12 ± 10% for mSUVs to 16% for 2kbv. Intersubject variability was high, with mean between-subject coefficients of variation ranging from 32 ± 13% for mSUVs to 45% for 2kbv. The highest pallidum:pons ratios of binding estimates were achieved by mSUV (4.2), spectral analysis-derived parametric maps (3.6), and 2kbv (3.6).

Conclusion

Quantification of CB₁ receptor availability using [¹¹C]MePPEP shows good to excellent reproducibility with several kinetic models and model-free analyses, whether applied on a region-of-interest or voxelwise basis. Simple mSUV measures were also reliable for most regions, but do not allow fully quantitative interpretation. [¹¹C]MePPEP PET is well placed as a tool to investigate CB₁-receptor mediated neurotransmission in health and disease.

•

[11C]MePPEP is a PET tracer for cannabinoid receptors (CB1R).

•

Extensive evaluation of [11C]MePPEP data quantification strategies in large sample

•

We highlight successful methods to quantify CB1R in regions of interest.

•

Highly reliable parametric maps (ICC 0.83 ± 0.03) allow whole-brain surveys.

•

Modified standard uptake values also reliable, without arterial input functions

Reproducibility and comparative validity of a food frequency questionnaire for Australian adults

BACKGROUND

Food frequency questionnaires (FFQ) are used in epidemiological studies to investigate the relationship between diet and disease. There is a need for a valid and reliable adult FFQ with a contemporary food list in Australia.

AIMS

To evaluate the reproducibility and comparative validity of the Australian Eating Survey (AES) FFQ in adults compared to weighed food records (WFRs).

METHODS

Two rounds of AES and three-day WFRs were conducted in 97 adults (31 males, median age and BMI for males of 44.9 years, 26.2 kg/m(2), females 41.3 years, 24.0 kg/m(2). Reproducibility was assessed over six months using Wilcoxon signed-rank tests and comparative validity was assessed by intraclass correlation coefficients (ICC) estimated by fitting a mixed effects model for each nutrient to account for age, sex and BMI to allow estimation of between and within person variance.

RESULTS

Reproducibility was found to be good for both WFR and FFQ since there were no significant differences between round 1 and 2 administrations. For comparative validity, FFQ ICCs were at least as large as those for WFR. The ICC of the WFR-FFQ difference for total energy intake was 0.6 (95% CI 0.43, 0.77) and the median ICC for all nutrients was 0.47, with all ICCs between 0.15 (%E from saturated fat) and 0.7 (g/day sugars).

CONCLUSIONS

Compared to WFR the AES FFQ is suitable for reliably estimating the dietary intakes of Australian adults across a wide range of nutrients.

Validation of trichloroacetic acid exposure via drinking water during pregnancy using a urinary TCAA biomarker

Disinfection by-product (DBP) exposure during pregnancy may be related to reduced fetal growth, but the evidence is inconclusive and improved DBP exposure assessment is required. The authors conducted a nested exposure study on a subset (n=39) of pregnant women in the Born in Bradford cohort to assess validity of TCAA exposure assessment based on tap water sampling and self-reported water-use; water-use questionnaire validity; and use of a one-time urinary TCAA biomarker. TCAA levels in urine and home tap water supply were quantified, and water use was measured via a questionnaire and 7-day diary, at 28 weeks gestation. Diary and urine measures were repeated later in pregnancy (n=14). TCAA level in home tap water supply was not correlated with urinary TCAA (0.18, P=0.29). Cold unfiltered tap water intake at home measured by questionnaire was correlated with urinary TCAA (0.44, P=0.007), but correlation was stronger still for cold unfiltered tap water intake reported over the 3 days prior to urine sampling (0.60, P<0.001). For unemployed women TCAA ingestion at home, derived from tap water sampling and self-reported water-use, correlated strongly with urinary TCAA (0.78, P<0.001), but for employed women the correlation was weak (0.31, P=0.20). Results suggest individual tap water intake is most influential in determining TCAA exposure variability in this cohort, and that TCAA ingestion at home is a valid proxy for TCAA exposure for unemployed women but less satisfactory for employed women.

Health related quality of life in patients with biliary atresia surviving with their native liver

OBJECTIVES

To quantify health related quality of life (HRQOL) of patients with biliary atresia with their native livers and compare them with healthy children and patients with biliary atresia post-liver transplant (LT) and to examine the relationship between HRQOL and medical variables.

STUDY DESIGN

A cross-sectional HRQOL study of patients with biliary atresia with their native livers (ages 2-25 years) was conducted and compared with healthy and post-LT biliary atresia samples using Pediatric Quality of Life Inventory 4.0 child self and parent proxy reports, a validated measure of physical/psychosocial functioning.

RESULTS

221 patients with biliary atresia with native livers (54% female, 67% white) were studied. Patient self and parent proxy reports showed significantly poorer HRQOL than healthy children across all domains (P < .001), particularly in emotional and psychosocial functioning. Child self and parent proxy HRQOL scores from patients with biliary atresia with their native livers and post-LT biliary atresia were similar across all domains (P = not significant). Child self and parent proxy reports showed moderate agreement across all scales, except social functioning (poor to fair agreement). On multivariate regression analysis, black race and elevated total bilirubin were associated with lower Total and Psychosocial HRQOL summary scores.

CONCLUSIONS

HRQOL in patients with biliary atresia with their native livers is significantly poorer than healthy children and similar to children with post-LT biliary atresia. These findings identify significant opportunities to optimize the overall health of patients with biliary atresia.

A systematic review of statistical methods used to test for reliability of medical instruments measuring continuous variables

OBJECTIVE(S)

Reliability measures precision or the extent to which test results can be replicated. This is the first ever systematic review to identify statistical methods used to measure reliability of equipment measuring continuous variables. This studyalso aims to highlight the inappropriate statistical method used in the reliability analysis and its implication in the medical practice.

MATERIALS AND METHODS

In 2010, five electronic databases were searched between 2007 and 2009 to look for reliability studies. A total of 5,795 titles were initially identified. Only 282 titles were potentially related, and finally 42 fitted the inclusion criteria.

RESULTS

The Intra-class Correlation Coefficient (ICC) is the most popular method with 25 (60%) studies having used this method followed by the comparing means (8 or 19%). Out of 25 studies using the ICC, only 7 (28%) reported the confidence intervals and types of ICC used. Most studies (71%) also tested the agreement of instruments.

CONCLUSION

This study finds that the Intra-class Correlation Coefficient is the most popular method used to assess the reliability of medical instruments measuring continuous outcomes. There are also inappropriate applications and interpretations of statistical methods in some studies. It is important for medical researchers to be aware of this issue, and be able to correctly perform analysis in reliability studies.