Application of a Web-based Tool for Quantitative Bias Analysis: The Example of Misclassification Due to Self-reported Body Mass Index

BACKGROUND

We describe the use of Apisensr, a web-based application that can be used to implement quantitative bias analysis for misclassification, selection bias, and unmeasured confounding. We apply Apisensr using an example of exposure misclassification bias due to use of self-reported body mass index (BMI) to define obesity status in an analysis of the relationship between obesity and diabetes.

METHODS

We used publicly available data from the National Health and Nutrition Examination Survey. The analysis consisted of: (1) estimating bias parameter values (sensitivity, specificity, negative predictive value, and positive predictive value) for self-reported obesity by sex, age, and race-ethnicity compared to obesity defined by measured BMI, and (2) using Apisensr to adjust for exposure misclassification.

RESULTS

The discrepancy between self-reported and measured obesity varied by demographic group (sensitivity range: 75%-89%; specificity range: 91%-99%). Using Apisensr for quantitative bias analysis, there was a clear pattern in the results: the relationship between obesity and diabetes was underestimated using self-report in all age, sex, and race-ethnicity categories compared to measured obesity. For example, in non-Hispanic White men aged 40-59 years, prevalence odds ratios for diabetes were 3.06 (95% confidence inerval = 1.78, 5.30) using self-reported BMI and 4.11 (95% confidence interval = 2.56, 6.75) after bias analysis adjusting for misclassification.

CONCLUSION

Apisensr is an easy-to-use, web-based Shiny app designed to facilitate quantitative bias analysis. Our results also provide estimates of bias parameter values that can be used by other researchers interested in examining obesity defined by self-reported BMI.

Differential Item Functioning in Reports of Delinquent Behavior Between Black and White Youth: Evidence of Measurement Bias in Self-Reports of Arrest in the Adolescent Brain Cognitive Development Study

Youth self-reports are a mainstay of delinquency assessment; however, making valid inferences about delinquency using these assessments requires equivalent measurement across groups of theoretical interest. We examined whether a brief 10-item delinquency measure exhibited measurement invariance across non-Hispanic White (n = 6,064) and Black (n = 1,666) youth (ages 10-11 years old) in the Adolescent Brain Cognitive Developmentsm Study (ABCD Study®). We detected differential item functioning (DIF) in two items. Black youth were more likely to report being arrested or picked up by police than White youth with the same score on the latent delinquency trait. Although multiple covariates (income, urgency, and callous-unemotional traits) reduced mean-level difference in overall delinquency, they were generally unrelated to the DIF in the Arrest item. However, the DIF in the Arrest item was reduced in size and no longer significant after adjusting for neighborhood safety. Results illustrate the importance of considering measurement invariance when using self-reported delinquency scores to draw inferences about group differences, and the utility of measurement invariance analyses for helping to identify mechanisms that contribute to group differences generally.

Remote Testing of Reading Comprehension in 8-Year-Old Children: Mode and Setting Effects

Proctored remote testing of cognitive abilities in the private homes of test-takers is becoming an increasingly popular alternative to standard psychological assessments in test centers or classrooms. Because these tests are administered under less standardized conditions, differences in computer devices or situational contexts might contribute to measurement biases that impede fair comparisons between test-takers. Because it is unclear whether cognitive remote testing might be a feasible assessment approach for young children, the present study (N = 1,590) evaluated a test of reading comprehension administered to children at the age of 8 years. To disentangle mode from setting effects, the children finished the test either in the classroom on paper or computer or remotely on tablets or laptops. Analyses of differential response functioning found notable differences between assessment conditions for selected items. However, biases in test scores were largely negligible. Only for children with below-average reading comprehension small setting effects between on-site and remote testing were observed. Moreover, response effort was higher in the three computerized test versions, among which, reading on tablets most strongly resembled the paper condition. Overall, these results suggest that, on average, even for young children remote testing introduces little measurement bias.

Different Responses to Social Capital Among Black People and White People: What Racial Differential Item Functioning Reveals for Racial Health Equity

Social capital has been conceptualized as features of social organization, such as networks, and norms that facilitate coordination and cooperation for mutual benefit. Because of long-standing anti-Black structural oppression in the United States, social capital may be associated with health differently for Black people than for other racial/ethnic groups. Our aim was to examine the psychometric properties of social capital indicators, comparing responses from Black and White people to identify whether there is differential item functioning (DIF) in social capital according to race. DIF examines how items are related to a latent construct and whether this relationship differs across groups such as different racial groups. We used data from respondents to the Southeastern Pennsylvania Household Health Survey in 2004, who lived in Philadelphia (n = 2,048), a city with a large Black population. We used item response theory analysis to test for racial DIF. We found DIF across the items, indicating measurement error, which could be related to the way these items were developed (i.e., based on cultural assumptions tested in mainstream White America). Hence, our findings underscore the need to interrogate the assumptions that underly existing social capital items through an equity-based lens, and to take corrective action when developing new items to ensure that they are racially and culturally congruent.

The biasing effects of selection and attrition on estimating the mean

Organizational and validation researchers often work with data that has been subjected to selection on the predictor and attrition on the criterion. These researchers often use the data observed under these conditions to estimate either the predictor or criterion's restricted population means. We show that the restricted means due to direct or indirect selection are a function of the population means plus the selection ratios. Thus, any difference between selected mean groups reflects the population difference plus the selection ratio difference. When there is also attrition on the criterion, the estimation of group differences becomes even more complicated. The effect of selection and attrition induces measurement bias when estimating the restricted population mean of either the predictor or criterion. A sample mean observed under selection and attrition does not estimate either the population mean or the restricted population mean. We propose several procedures under normality that yield unbiased estimates of the mean. The procedures focus on correcting the effects of selection and attrition. Each procedure was evaluated with a Monte Carlo simulation to ascertain its strengths and weaknesses. Given appropriate sample size and conditions, we show that these procedures yield unbiased estimators of the restricted and unrestricted population means for both predictor and criterion. We also show how our findings have implications for replicating selected group differences.

A Thermal Skin Model for Comparing Contact Skin Temperature Sensors and Assessing Measurement Errors

To improve the measurement and subsequent use of human skin temperature (Tsk) data, there is a need for practical methods to compare Tsk sensors and to quantify and better understand measurement error. We sought to develop, evaluate, and utilize a skin model with skin-like thermal properties as a tool for benchtop Tsk sensor comparisons and assessments of local temperature disturbance and sensor bias over a range of surface temperatures. Inter-sensor comparisons performed on the model were compared to measurements performed in vivo, where 14 adult males completed an experimental session involving rest and cycling exercise. Three types of Tsk sensors (two of them commercially available and one custom made) were investigated. Skin-model-derived inter-sensor differences were similar (within ±0.4 °C) to the human trial when comparing the two commercial Tsk sensors, but not for the custom Tsk sensor. Using the skin model, all surface Tsk sensors caused a local temperature disturbance with the magnitude and direction dependent upon the sensor and attachment and linearly related to the surface-to-environment temperature gradient. Likewise, surface Tsk sensors also showed bias from both the underlying disturbed surface temperature and that same surface in its otherwise undisturbed state. This work supports the development and use of increasingly realistic benchtop skin models for practical Tsk sensor comparisons and for identifying potential measurement errors, both of which are important for future Tsk sensor design, characterization, correction, and end use.

Temporal Alignment of Dual Monitor Accelerometry Recordings

Combining accelerometry from multiple independent activity monitors worn by the same subject have gained widespread interest with the assessment of physical activity behavior. However, a difference in the real time clock accuracy of the activity monitor introduces a substantial temporal misalignment with long duration recordings which is commonly not considered. In this study, a novel method not requiring human interaction is described for the temporal alignment of triaxial acceleration measured with two independent activity monitors and evaluating the performance with the misalignment manually identified. The method was evaluated with free-living recordings using both combined wrist/hip (n = 9) and thigh/hip device (n = 30) wear locations, and descriptive data on initial offset and accumulated day 7 drift in a large-scale population-based study (n = 2513) were calculated. The results from the Bland–Altman analysis show good agreement between the proposed algorithm and the reference suggesting that the described method is valid for reducing the temporal misalignment and thus reduce the measurement error with aggregated data. Applying the algorithm to the n = 2513 samples worn for 7-days suggest a wide and substantial issue with drift over time when each subject wears two independent activity monitors.

Differential frequency in imaging-based outcome measurement: Bias in real-world oncology comparative-effectiveness studies

Background

Comparative‐effectiveness studies using real‐world data (RWD) can be susceptible to surveillance bias. In solid tumor oncology studies, analyses of endpoints such as progression‐free survival (PFS) are based on progression events detected by imaging assessments. This study aimed to evaluate the potential bias introduced by differential imaging assessment frequency when using electronic health record (EHR)‐derived data to investigate the comparative effectiveness of cancer therapies.

Methods

Using a nationwide de‐identified EHR‐derived database, we first analyzed imaging assessment frequency patterns in patients diagnosed with advanced non‐small cell lung cancer (aNSCLC). We used those RWD inputs to develop a discrete event simulation model of two treatments where disease progression was the outcome and PFS was the endpoint. Using this model, we induced bias with differential imaging assessment timing and quantified its effect on observed versus true treatment effectiveness. We assessed percent bias in the estimated hazard ratio (HR).

Results

The frequency of assessments differed by cancer treatment types. In simulated comparative‐effectiveness studies, PFS HRs estimated using real‐world imaging assessment frequencies differed from the true HR by less than 10% in all scenarios (range: 0.4% to −9.6%). The greatest risk of biased effect estimates was found comparing treatments with widely different imaging frequencies, most exaggerated in disease settings where time to progression is very short.

Conclusions

This study provided evidence that the frequency of imaging assessments to detect disease progression can differ by treatment type in real‐world patients with cancer and may induce some bias in comparative‐effectiveness studies in some situations.

The problem of measurement bias in comparing selected subgroups

Estimates of subgroup differences are routinely used as part of a comprehensive validation system, and these estimates serve a critical role, including evaluating adverse impact. Unfortunately, under direct range restriction, a selected mean ( μ̂t' ) is a biased estimator of the population mean μx as well as the selected true score mean μt' . This is due partly to measurement bias. This bias, as we show, is a factor of the selection ratio, the reliability of the measure, and the variance of the distribution. This measurement bias renders a subgroup comparison questionable when the subgroups have different selection ratios. The selected subgroup comparison is further complicated by the fact that the subgroup variances will be unequal in most situations where the selection ratios are not equal. We address these problems and present a corrected estimate of the mean difference, as well as an estimate of Cohen's d* that estimates the true score difference between two selected populations, (μtA'-μtB')/σt . In addition, we show that the measurement bias is not present under indirect range restriction. Thus, the observed selected mean μ̂t' is an unbiased estimator of selected true score mean μty' . However, it is not an unbiased estimator of the population mean μy . These results have important implications for selection research, particularly when validating instruments.

Measurement bias and error correction in a two-stage estimation for multilevel IRT models

Among current state-of-the-art estimation methods for multilevel IRT models, the two-stage divide-and-conquer strategy has practical advantages, such as clearer definition of factors, convenience for secondary data analysis, convenience for model calibration and fit evaluation, and avoidance of improper solutions. However, various studies have shown that, under the two-stage framework, ignoring measurement error in the dependent variable in stage II leads to incorrect statistical inferences. To this end, we proposed a novel method to correct both measurement bias and measurement error of latent trait estimates from stage I in the stage II estimation. In this paper, the HO-IRT model is considered as the measurement model, and a linear mixed effects model on overall (i.e., higher-order) abilities is considered as the structural model. The performance of the proposed correction method is illustrated and compared via a simulation study and a real data example using the National Educational Longitudinal Survey data (NELS 88). Results indicate that structural parameters can be recovered better after correcting measurement biases and errors.

When Does Differential Item Functioning Matter for Screening? A Method for Empirical Evaluation

When items in a screening measure exhibit differential item functioning (DIF) across groups (e.g., males vs. females), DIF might affect which individuals are "caught" in the screening. This phenomenon is common, but DIF detection procedures do not typically provide guidance on whether the presence of DIF will meaningfully affect screening accuracy. Millsap and Kwok proposed a method to quantify the impact of DIF on screening accuracy, but their approach had limitations that prevent its use in scenarios where items are discrete. We extend the Millsap and Kwok procedure to accommodate discrete items and provide R functions to apply the procedure to the user's own data. We illustrate our approach using published screening information and evaluate the proposed methodology with a small simulation study. Overall, we encourage researchers to use empirical methods to evaluate the extent to which the presence of DIF in a screening measure materially affects screening performance.

Effects of freezing on length and mass measurements of Atlantic cod Gadus morhua in the Baltic Sea

An aggregated sample of 925 Atlantic cod Gadus morhua collected by four countries in different regions of the Baltic Sea during different seasons were measured (total length, L_T = 161-890 mm and weighed (mass, M = 45-6900 g) both before freezing and after defrosting. The cod were found to decrease significantly in both L_T and M following death and frozen storage. There was an average (±SD) change in L_T of -2.91% (±0.05%) following freezing, independent of starting L_T . Total M changed by -2.65% (±0.14%), independent of starting mass. Shrinkage of L_T and M did not differ significantly between 1 and 4 months frozen storage, though L_T shrinkage was significantly greater after 1 or 4 months in the freezer compared with after 5 days. There was significant variation in L_T and M shrinkage between regions of capture. A significant negative relationship between condition of cod and L_T or M change was also observed. Equations to back-calculate fresh L_T and M from thawed L_T , M and standard length (L_S ), gutted L_T , gutted L_S and gutted M are provided.

Using fathers as a negative control exposure: Implications of measurement bias

Negative control exposure analysis is a very effective tool in evaluating the effect of unmeasured confounding in observational epidemiological studies. Several biases, including recall bias, time-varying confounding factors, measurement bias and so on, can affect the credibility of negative control exposure analysis for causal interpretations. The article focuses on the implications of differential measurement error across exposed group and negative controls to causal interpretations on negative control exposure analysis.

Timing of nest vegetation measurement may obscure adaptive significance of nest-site characteristics: A simulation study

Advances in understanding avian nesting ecology are hindered by a prevalent lack of agreement between nest‐site characteristics and fitness metrics such as nest success. We posit this is a result of inconsistent and improper timing of nest‐site vegetation measurements. Therefore, we evaluated how the timing of nest vegetation measurement influences the estimated effects of vegetation structure on nest survival. We simulated phenological changes in nest‐site vegetation growth over a typical nesting season and modeled how the timing of measuring that vegetation, relative to nest fate, creates bias in conclusions regarding its influence on nest survival. We modeled the bias associated with four methods of measuring nest‐site vegetation: Method 1—measuring at nest initiation, Method 2—measuring at nest termination regardless of fate, Method 3—measuring at nest termination for successful nests and at estimated completion for unsuccessful nests, and Method 4—measuring at nest termination regardless of fate while also accounting for initiation date. We quantified and compared bias for each method for varying simulated effects, ranked models for each method using AIC, and calculated the proportion of simulations in which each model (measurement method) was selected as the best model. Our results indicate that the risk of drawing an erroneous or spurious conclusion was present in all methods but greater with Method 2 which is the most common method reported in the literature. Methods 1 and 3 were similarly less biased. Method 4 provided no additional value as bias was similar to Method 2 for all scenarios. While Method 1 is seldom practical to collect in the field, Method 3 is logistically practical and minimizes inherent bias. Implementation of Method 3 will facilitate estimating the effect of nest‐site vegetation on survival, in the least biased way, and allow reliable conclusions to be drawn.

Religiosity in black and white older Americans: Measure adaptation, psychometric validation, and racial difference

Racial difference of religiosity in a heterogeneous older population had long been a focal point of gerontological research. However, most religiosity measures were developed from homogenous sample, few underwent rigorous psychometric validation, and studies on racial difference of religiosity had been obstructed. This cross-sectional study adapted a religiosity measure originally designed for blacks only to a heterogeneous older population of blacks and whites, validated its psychometric properties, and examined racial difference of religiosity. Based on qualitative research of concepts, intensive literature review, and abundant experiences in this field, we adapted the original measure. Then, using the data collected from a survey of 196 black and white Americans 55 years and older in Charlotte, North Carolina, we investigated full-scale psychometric properties of the adapted measure at the item-, domain-, and measure- level. These psychometric validations included item analysis, item-scale correlations, correlation matrix among items, confirmatory factor analysis (CFA) to determine if the original factor structure held after adaptation, and reliability analysis using Cronbach's alpha. Finally, using Multiple Indicators and MultIple Causes (MIMIC) models, we examined racial difference of religiosity through regression with latent variables, while potential measurement bias by race through differential item functioning (DIF) was adjusted in the MIMIC models. In result, we successfully adapted the original 12-item religiosity measure for blacks into an 8-item version for blacks and whites. Although sacrificed few reliability for brevity, the adapted measure demonstrated sound psychometric properties, and retained the original factor structure. We also found racial differences of religiosity in all three domains of the measure, even after adjustment of the detected measurement biases in two domains. In conclusion, the original measure can be adapted to and validated for a heterogeneous older population of blacks and whites. Although the adapted measure can be used to measure the three domains of religiosity in blacks and whites, the observed racial differences of religiosity need to be adjusted for measurement biases before meaningful comparisons.

On the power of the test for cluster bias

Cluster bias refers to measurement bias with respect to the clustering variable in multilevel data. The absence of cluster bias implies absence of bias with respect to any cluster-level (level 2) variable. The variables that possibly cause the bias do not have to be measured to test for cluster bias. Therefore, the test for cluster bias serves as a global test of measurement bias with respect to any level 2 variable. However, the validity of the global test depends on the Type I and Type II error rates of the test. We compare the performance of the test for cluster bias with the restricted factor analysis (RFA) test, which can be used if the variable that leads to measurement bias is measured. It appeared that the RFA test has considerably more power than the test for cluster bias. However, the false positive rates of the test for cluster bias were generally around the expected values, while the RFA test showed unacceptably high false positive rates in some conditions. We conclude that if no significant cluster bias is found, still significant bias with respect to a level 2 violator can be detected with an RFA model. Although the test for cluster bias is less powerful, an advantage of the test is that the cause of the bias does not need to be measured, or even known.

Functioning of cannabis abuse and dependence criteria across two different countries: the United States and The Netherlands

BACKGROUND

Cross-national differences could affect the likelihood of endorsement of DSM cannabis abuse and dependence criteria. The present study examines whether cannabis abuse and dependence criteria function differently across U.S. and Dutch cannabis users.

METHOD

Data on lifetime endorsement of DSM-IV cannabis abuse/dependence criteria were utilized from U.S. cannabis users who participated in the National Epidemiological Survey on Alcohol and Related Conditions (NESARC) and from Dutch cannabis users who participated in the Zuid-Holland study. In total, 1,568 cannabis users participated in the NESARC sample, and 359 cannabis users participated in the Zuid-Holland sample. The DSM-IV cannabis abuse/dependence criteria as well as cannabis withdrawal were determined using face-to-face computer-assisted personal interviews.

RESULTS

Using Restricted Factor Analysis with Latent Moderated Structures, the cannabis abuse/dependence criteria legal problems (β = -0.43), failed quit attempts (β = -1.09), use despite problems (β = -0.32), and withdrawal (β = -0.53) showed measurement bias, and were more likely to be endorsed by U.S. than by Dutch cannabis users. Also, men were more likely than women to endorse the criteria hazardous use (β = -0.27), legal problems (β = -0.49) and tolerance (β = -0.20). Findings on failed quit attempts and withdrawal were replicated in matched subsamples, while results on legal problems (country and gender) were partly replicated.

CONCLUSIONS

Several CUD criteria showed measurement bias across two countries and between males and females. Therefore, differences between countries and gender in prevalence rates of CUD should be regarded with caution.

Testing strong factorial invariance using three-level structural equation modeling

Within structural equation modeling, the most prevalent model to investigate measurement bias is the multigroup model. Equal factor loadings and intercepts across groups in a multigroup model represent strong factorial invariance (absence of measurement bias) across groups. Although this approach is possible in principle, it is hardly practical when the number of groups is large or when the group size is relatively small. Jak et al. (2013) showed how strong factorial invariance across large numbers of groups can be tested in a multilevel structural equation modeling framework, by treating group as a random instead of a fixed variable. In the present study, this model is extended for use with three-level data. The proposed method is illustrated with an investigation of strong factorial invariance across 156 school classes and 50 schools in a Dutch dyscalculia test, using three-level structural equation modeling.

What's hampering measurement invariance: detecting non-invariant items using clusterwise simultaneous component analysis

The issue of measurement invariance is ubiquitous in the behavioral sciences nowadays as more and more studies yield multivariate multigroup data. When measurement invariance cannot be established across groups, this is often due to different loadings on only a few items. Within the multigroup CFA framework, methods have been proposed to trace such non-invariant items, but these methods have some disadvantages in that they require researchers to run a multitude of analyses and in that they imply assumptions that are often questionable. In this paper, we propose an alternative strategy which builds on clusterwise simultaneous component analysis (SCA). Clusterwise SCA, being an exploratory technique, assigns the groups under study to a few clusters based on differences and similarities in the component structure of the items, and thus based on the covariance matrices. Non-invariant items can then be traced by comparing the cluster-specific component loadings via congruence coefficients, which is far more parsimonious than comparing the component structure of all separate groups. In this paper we present a heuristic for this procedure. Afterwards, one can return to the multigroup CFA framework and check whether removing the non-invariant items or removing some of the equality restrictions for these items, yields satisfactory invariance test results. An empirical application concerning cross-cultural emotion data is used to demonstrate that this novel approach is useful and can co-exist with the traditional CFA approaches.

Misclassification in assessment of diabetogenic risk using electronic health records

PURPOSE

Suspected diabetogenic effects or drug indication may increase testing for diabetes mellitus (DM), resulting in measurement bias when evaluating diabetogenic drug effects. We sought to evaluate the validity of electronic health record data in determining DM risk.

METHODS

We used time-dependent Cox proportional hazard models within a retrospective cohort design to assess associations between use of antihypertensives, statins, atypical antipsychotics, and antidepressants, and two endpoints: (i) DM onset defined as fasting blood glucose (BG) ≥126 mg/dl, random BG ≥200 mg/dl, HbA1c ≥7.0%, or antidiabetic drug initiation; and (ii) first negative DM test. We used Poisson regression to assess the influence of these drugs on DM testing rates. Patients aged 35-64 years enrolled in Kaiser Permanente Northwest between 1997 and 2010 entered the cohort at the first negative BG test after ≥6 months without manifest DM.

RESULTS

All drug classes showed significant associations not only with DM onset but also with first negative BG test and with DM testing rates. Antipsychotics had the greatest diabetogenic risk (adjusted hazard ratio [HR] = 1.73 [1.44-2.08]), the greatest propensity for a first negative test (adjusted HR = 1.87 [1.74-2.01]), and the highest testing rate (adjusted rate ratio = 1.76 [1.72-1.81]. Although renin-angiotensin system blockers and calcium channel blockers have shown no diabetogenic risk in clinical trials, both were associated with DM (HR = 1.19 [1.12-1.26] and 1.27 [1.17-1.38]), a negative glucose test (1.38 [1.35-1.41] and 1.24 [1.20-1.28]), and increased testing rates (rate ratio = 1.26 [1.24-1.27] and 1.27 [1.25-1.28]).

CONCLUSION

Caution should be used when diabetogenic risk is evaluated using data that rely on DM testing in general practice.

Within-hospital readmission: an indicator of readmission after discharge from psychiatric hospitalization

OBJECTIVE

Readmission after psychiatric hospitalization is widely used as a quality of care indicator by government funding agencies, policy-makers, and hospitals deciding on clinical priorities. Readmission rates are calculated accurately to allow these varied groups to correctly translate the knowledge into appropriate, tangible outcomes. We aimed to assess how well hospital readmission rates, calculated using only readmissions to the discharging institution, can approximate actual readmission rates.

METHOD

We used administrative data sources to identify patients with a mental health discharge in the province of Ontario (2008-2011). We identified mental health readmissions within 30 and 90 days of discharge occurring to the hospital from which the patient was discharged (within-hospital readmissions), and compared readmission rates using only within-hospital admissions with actual readmission rates.

RESULTS

The percentage of readmissions occurring to the discharging institution ranged from 39% to 89% (median 73%) and from 37% to 86% (median 70%) for 30- and 90-day readmissions, respectively. Using only within-hospital readmissions to rank hospitals by their readmission rates, only 56% of hospitals for 30-day readmissions and 50% for 90-day readmissions were ranked in the same quartile as when actual readmission rates were used.

CONCLUSIONS

These findings highlight the importance of measuring psychiatric readmissions at the system level, particularly for hospitals with lower discharge volumes. As well, the high likelihood that multiple hospitals are involved in the hospital-based care of people who require readmission requires consideration at clinical and policy levels.

Using the Criterion-Predictor Factor Model to Compute the Probability of Detecting Prediction Bias with Ordinary Least Squares Regression

The study of prediction bias is important and the last five decades include research studies that examined whether test scores differentially predict academic or employment performance. Previous studies used ordinary least squares (OLS) to assess whether groups differ in intercepts and slopes. This study shows that OLS yields inaccurate inferences for prediction bias hypotheses. This paper builds upon the criterion-predictor factor model by demonstrating the effect of selection, measurement error, and measurement bias on prediction bias studies that use OLS. The range restricted, criterion-predictor factor model is used to compute Type I error and power rates associated with using regression to assess prediction bias hypotheses. In short, OLS is not capable of testing hypotheses about group differences in latent intercepts and slopes. Additionally, a theorem is presented which shows that researchers should not employ hierarchical regression to assess intercept differences with selected samples.

Using structural equation modeling to detect response shift in performance and health-related quality of life scores of multiple sclerosis patients

PURPOSE

To illustrate how structural equation modeling (SEM) can be used for response shift detection with random measurement occasions and health state operationalized as fixed group membership (Study 1) or with fixed measurement occasions and health state operationalized as time-varying covariates (Study 2).

METHODS

In Study 1, we explored seven items of the Performance Scales measuring physical and mental aspects of perceived disability of 771 stable, 629 progressive, and 1,552 relapsing MS patients. Time lags between the three measurements varied and were accounted for by introducing time since diagnosis as an exogenous variable. In Study 2, we considered the SF-12 scales measuring physical and mental components of HRQoL of 1,767 patients. Health state was accounted for by exogenous variables relapse (yes/no) and symptoms (worse/same/better).

RESULTS

In Study 1, progressive and relapsing patients reported greater disability than stable patients but little longitudinal change. Some response shift was found with stable and relapsing patients. In Study 2, relapse and symptoms were associated with HRQoL, but no change and only little response shift was found.

CONCLUSIONS

While small response shifts were found, they had little impact on the evaluation of true change in performance and HRQoL.

Exploring measurement biases associated with esophageal Doppler monitoring in critically ill patients in intensive care unit

BACKGROUND:

Esophageal Doppler monitoring (EDM) is utilized in numerous clinical settings. This study examines the relationship between pulmonary artery catheter (PAC) and EDM-derived hemodynamic parameters, concentrating on gender- and age-related EDM measurement biases.

MATERIALS AND METHODS:

Prospective study of EDM use in ventilated surgical ICU patients. Parameters examined included demographics, diagnosis, resuscitation endpoints, cardiac output (CO) and stroke volume from both devices, number of personnel and time needed to place equipment, time to data acquisition, duration of use, complications of placement.

RESULTS:

Fifteen patients (11 men, 4 women, mean age 47 years) were included. Most common diagnoses included trauma (7/15) and sepsis (4/15). Insertion time and time to data acquisition were shorter for EDM than for PAC (P<0.001). The EDM required an average of 1.1 persons to place (2.4 for PAC, P=0.002). Mean EDM utilization time was 12.4 h. There was a fair CO correlation between EDM and PAC (r = 0.647, P<0.001). Overall, the EDM underestimated CO relative to PAC (bias -1.42 ± 2.08, 95% CI: -5.58-2.74), with more underestimation in women (mean bias difference of -1.16, P<0.001). No significant age-related measurement bias differences between PAC and EDM were noted. Significant reductions in lactate and norepinephrine requirement were noted following EDM monitoring periods.

CONCLUSIONS:

This study found that the EDM significantly underestimated cardiac output in women when compared to PAC. Clinicians should be aware of this measurement bias when making therapeutic decision based on EDM data. Significant reductions in lactate and norepinephrine requirement during EDM monitoring periods support the clinical usefulness of EDM technology.

Cross-cultural invalidity of alcohol dependence measurement across Hispanics and Caucasians in 2001 and 2002

AIMS

Do assessments of alcohol dependence demonstrate similarly validity across Hispanics and non-Hispanic Caucasians? This investigation examined this question.

METHOD

It employed confirmatory factor analyses for ordered-categorical measures to search for measurement bias on the AUDADIS, a standardized measure of alcohol dependence, across Hispanic (n=4819) and non-Hispanic Caucasians (n=16, 109) in a nationally representative survey of alcohol use in the United States conducted in 2001 and 2002.

MEASUREMENT

Analyses considered whether 27 items operationalizing the DSM-IV alcohol dependence construct provided equivalent measurement.

FINDINGS AND CONCLUSIONS

Nine items revealed statistically significant bias, suggesting strong caution regarding the cross-ethnic validity of alcohol dependence. Sensitivity analyses established that item level differences erroneously impact alcohol dependence estimates among the 2001-2002 US Hispanic population. Biased measurement underestimates differences between Hispanics and non-Hispanic Caucasians, underestimates Hispanics' true use levels, and falsely minimizes current increases in drinking behavior evidenced among Hispanics. Findings urge improved public health efforts among the Hispanic community and underscore the necessity for cultural sensitivity when generalizing measures and constructs developed in the majority to Hispanic individuals.