Reducing bias in trials from reactions to measurement: the MERIT study including developmental work and expert workshop

BACKGROUND

Measurement can affect the people being measured; for example, asking people to complete a questionnaire can result in changes in behaviour (the 'question-behaviour effect'). The usual methods of conduct and analysis of randomised controlled trials implicitly assume that the taking of measurements has no effect on research participants. Changes in measured behaviour and other outcomes due to measurement reactivity may therefore introduce bias in otherwise well-conducted randomised controlled trials, yielding incorrect estimates of intervention effects, including underestimates.

OBJECTIVES

The main objectives were (1) to promote awareness of how and where taking measurements can lead to bias and (2) to provide recommendations on how best to avoid or minimise bias due to measurement reactivity in randomised controlled trials of interventions to improve health.

METHODS

We conducted (1) a series of systematic and rapid reviews, (2) a Delphi study and (3) an expert workshop. A protocol paper was published [Miles LM, Elbourne D, Farmer A, Gulliford M, Locock L, McCambridge J, et al. Bias due to MEasurement Reactions In Trials to improve health (MERIT): protocol for research to develop MRC guidance. Trials 2018;19:653]. An updated systematic review examined whether or not measuring participants had an effect on participants' health-related behaviours relative to no-measurement controls. Three new rapid systematic reviews were conducted to identify (1) existing guidance on measurement reactivity, (2) existing systematic reviews of studies that have quantified the effects of measurement on outcomes relating to behaviour and affective outcomes and (3) experimental studies that have investigated the effects of exposure to objective measurements of behaviour on health-related behaviour. The views of 40 experts defined the scope of the recommendations in two waves of data collection during the Delphi procedure. A workshop aimed to produce a set of recommendations that were formed in discussion in groups.

RESULTS

Systematic reviews - we identified a total of 43 studies that compared interview or questionnaire measurement with no measurement and these had an overall small effect (standardised mean difference 0.06, 95% confidence interval 0.02 to 0.09; n = 104,096, I² = 54%). The three rapid systematic reviews identified no existing guidance on measurement reactivity, but we did identify five systematic reviews that quantified the effects of measurement on outcomes (all focused on the question-behaviour effect, with all standardised mean differences in the range of 0.09-0.28) and 16 studies that examined reactive effects of objective measurement of behaviour, with most evidence of reactivity of small effect and short duration. Delphi procedure - substantial agreement was reached on the scope of the present recommendations. Workshop - 14 recommendations and three main aims were produced. The aims were to identify whether or not bias is likely to be a problem for a trial, to decide whether or not to collect further quantitative or qualitative data to inform decisions about if bias is likely to be a problem, and to identify how to design trials to minimise the likelihood of this bias.

LIMITATION

The main limitation was the shortage of high-quality evidence regarding the extent of measurement reactivity, with some notable exceptions, and the circumstances that are likely to bring it about.

CONCLUSION

We hope that these recommendations will be used to develop new trials that are less likely to be at risk of bias.

FUTURE WORK

The greatest need is to increase the number of high-quality primary studies regarding the extent of measurement reactivity.

STUDY REGISTRATION

The first systematic review in this study is registered as PROSPERO CRD42018102511.

FUNDING

Funded by the Medical Research Council UK and the National Institute for Health Research as part of the Medical Research Council-National Institute for Health Research Methodology Research Programme.

Fatigue Measurements in Systemic Lupus Erythematosus

OBJECTIVE

Fatigue is a frequent, disabling issue in systemic lupus erythematosus (SLE). It is, however, difficult to quantify. The Ad Hoc Committee on SLE Response Criteria for Fatigue in 2007 recommended using the Krupp Fatigue Severity Scale (FSS). Since then, the Functional Assessment of Chronic Illness Therapy (FACIT)-Fatigue Scale has also been validated in SLE. We performed a review of instruments used to measure fatigue in adult SLE patients from 2007 onward.

METHODS

We searched PubMed, Medline, and Embase (January 2008-October 2017), identifying clinical trials and observational studies in adult SLE, where fatigue was a specifically measured outcome. All English and French studies were reviewed to determine fatigue measures and results.

RESULTS

Thirty-seven studies met inclusion criteria. Eight scales were used. The visual analog scale (VAS), FSS, and FACIT-Fatigue Scale were most frequent. FSS was the most often used instrument in both clinical trials and observational studies. Twenty-five of the 37 studies demonstrated a difference in fatigue that was statistically significant and clinically meaningful. Of the 12 studies that did not, 6 used FSS, 3 used VAS, 2 used the Multidimensional Assessment of Fatigue, and 1 used the Brief Fatigue Index. All 6 studies using the FACIT-Fatigue Scale detected clinically meaningful and statistically significant differences.

CONCLUSION

VAS, FSS, and FACIT-Fatigue Scale were the most frequently used instruments in adult SLE studies from 2008 to 2017. Many studies detected clinically important changes in fatigue. Fatigue remains a key measure in both clinical trials and observational SLE studies.

Measuring empathy in undergraduate law students: Examining the factorial validity of the Jefferson Scale of Empathy - Law Students (JSE-LS)

Several scholars have hypothesised a link between empathy and a range of important outcomes for law students including well-being, mental health and the development of effective client-lawyer relationships. However, few studies have examined these claims empirically. Empirical investigation of empathy among law students requires effective methods of measuring empathy. The present study sought to examine an instrument designed specifically to measure empathy among law students - the Jefferson Empathy Scale - Law Students (JSE-LS). The study involved examining the internal consistency and factor structure of the instrument using a sample of 276 Australian undergraduate law students. The study found that a four-factor solution was optimal for the dataset. Two of the factors were readily interpretable with previous literature, however the remaining two factors were unstable, suggesting the need for further revision of the instrument. Recommendations for revising the JSE-LS to better measure empathy are discussed.

Double burden or double counting of child malnutrition? The methodological and theoretical implications of stuntingoverweight in low and middle income countries

BACKGROUND

There is increasing concern at research and policy levels about the double burden of child malnutrition (DBCM)-with stunting and overweight found across different groups of children. Despite some case studies suggesting that stunting and overweight can occur concurrently in children, here known as 'stuntingoverweight', and major drives to reduce all forms of malnutrition in low and middle income countries (LMICs), stuntingoverweight is continually overlooked. This research evidences the prevalence of stuntingoverweight across LMICs, exploring the theoretical and methodological implications of failing to acknowledge this form of malnutrition.

METHODS

Prevalence estimates of stuntingoverweight are constructed from 79 LMICs with nationally representative anthropometric survey data. Stunting and overweight estimates are amended to exclude stuntedoverweight children. These estimates are compared with those published in the Joint Child Malnutrition Estimates (JMEs)-evidencing overestimation and double counting of stuntedoverweight children.

RESULTS

Children can be concurrently stunted and overweight. Stuntedoverweight children are found in all LMICs, from 0.3% to 11.7% of under-fives and are included in both stunting and overweight rates. Analysed together, this leads to double counting of stuntedoverweight children. This artificial inflation of stunting and overweight rates can give a false impression of a DBCM, obscuring the true diversity of malnutrition present. Over 10 million children are stuntedoverweight in the world.

CONCLUSIONS

Stuntingoverweight is a newly recognised, understudied phenomenon. Affected children are included in both stunting and overweight prevalence estimates, introducing unobserved heterogeneity to both individual-level and population-level research and double counting to population-level research. Overlooking stuntedoverweight children has great implications for methodology, theory, policies, programmes and the health of affected children.

Harmonising summary measures of population health using global survey instruments

Summary measures of population health—health expectancies in particular—have become a standard for quantifying and monitoring population health. To date, cross-national comparability of health expectancies is limited, except within the European Union (EU). To advance international comparability, the European Joint Action on Healthy Life Years (JA: EHLEIS) set up an international working group. The working group discussed the conceptual basis of summary measures of population health and made suggestions for the development of comparable health expectancies to be used across the EU and Organisation for Economic Co-operation and Development (OECD) members. In this paper, which summarises the main results, we argue that harmonised health data needed for health expectancy calculation can best be obtained from ‘global’ survey measures, which provide a snapshot of the health situation using 1 or a few survey questions. We claim that 2 global measures of health should be pursued for their high policy relevance: a global measure of participation restriction and a global measure of functional limitation. We finally provide a blueprint for the future development and implementation of the 2 global measures. The blueprint sets the basis for subsequent international collaboration, having as a core group Member States of the EU, the USA and Japan. Other countries, in particular OECD members, are invited to join the initiative.

Trajectories of maternal gestational weight gain and child cognition assessed at 5 years of age in a prospective cohort study

BACKGROUND

There has been concern that low gestational weight gain may cause poor fetal neurodevelopment.

METHODS

The association between maternal weight gain and child IQ was examined using serial antenatal weight measurements (median 12) from a prospective cohort of non-obese Scandinavian women (1986-1988). Linear mixed models with piecewise regression were used to estimate participants' (n=552) trimester-specific average rate of weight gain. Linear regression was used to assess the association between weight gain and children's (n=344) full-scale, performance and verbal IQ measured at age 5 using the Wechsler Preschool and Primary Scales of Intelligence-Revised.

RESULTS

Children born to mothers who gained below versus within the 2nd trimester 2009 recommendations tended to have lower IQ scores (Full-scale: 106.6 (SD 15.1) vs 110.2 (15.2), p=0.04; verbal: 102.5 (14.3) vs 105.0 (14.9), p=0.10; performance: 109.5 (15.4) vs 113.4 (14.5), p=0.03). After adjustment there were no differences in child IQ by weight gain adequacy (full-scale: βbelow=-1.1 (95% CI -5.1 to 2.9), βabove=1.5 (-3.8 to 6.8); verbal: βbelow=-0.2 (-3.1 to 2.6), βabove=1.8 (-3.6 to 7.3); performance βbelow=-1.2 (-4.6 to 2.2), βabove=1.0 (-4.6 to 6.7)). No differences were observed based on 3rd trimester adequacy. No differences were observed in IQ scores by quintile of weight gain for any trimester, particularly after adjustment for maternal IQ.

CONCLUSIONS

Our findings are reassuring that among normal weight women, pregnancy weight gain is not associated with child cognitive development. Further investigation should be conducted in contemporary cohorts that also include obese mothers, who are at the greatest risk for low weight gain.

Sleep duration and health among older adults: associations vary by how sleep is measured

BACKGROUND

Cohort studies have found that short and long sleep are both associated with worse outcomes, compared with intermediate sleep times. While demonstrated biological mechanisms could explain health effects for short sleep, long-sleep risk is puzzling. Most studies reporting the U shape use a single question about sleep duration, a measurement method that does not correlate highly with objectively measured sleep. We hypothesised that the U shape, especially the poor outcomes for long sleepers, may be an artefact of how sleep is measured.

METHODS

We examined the cross-sectional prevalence of fair/poor health by sleep hour categories (≤ 6, ≤ 7, ≤ 8, ≤ 9, > 9 h) in a national US sample of adults aged 62-90 that included several types of sleep measures (n = 727). Survey measures were: a single question; usual bedtimes and waking times; and a 3-day sleep log. Actigraphy measures were the sleep interval and total sleep time. Fair/poor health was regressed on sleep hour categories adjusted for demographics, with tests for both linear trend and U shape.

RESULTS

Adjusted OR of fair/poor health across sleep hour categories from the single question were 4.6, 2.2, referent (8 h), 1.8 and 6.9. There was high prevalence of fair/poor health for ≤ 6 h for all sleep measures, but the long-sleep effect was absent for sleep logs and actigraphy measures.

CONCLUSIONS

Associations between long sleep and poor health may be specific to studies measuring sleep with survey questions. As cohorts with actigraphy mature, our understanding of how sleep affects health may change.

The use of parental occupation in adolescent health surveys. An application of ISCO-based measures of occupational status

BACKGROUND

Recent research has emphasised that the challenge in researching socioeconomic differences in adolescent health cross-nationally lies in providing valid and comparable measures of socioeconomic position (SEP) across regions. This study aims to examine measures of occupational status derived from the International Standard Classification of Occupations (ISCO), alongside commonly used affluence measures in association with adolescent self-rated health (SRH).

METHODS

Data were from the 2005/2006 'Health Behaviour in School-aged Children study' (HBSC); 27 649 individuals aged 11, 13 and 15 years from Germany, Macedonia, Norway, Turkey, Wales and Scotland. Three occupational scales were compared: the International Socioeconomic Index of Occupational Status (ISEI), the Standard International Occupational Prestige Scale (SIOPS) and the Erikson-Goldthorpe-Portocarero class categories (EGP). Correlation analyses compared these occupational scales with the family affluence scale (FAS) and a family well-off measure, while logistic regression assessed the association between occupational scales and poor SRH. Multiple imputation techniques investigated possible bias arising from parental occupation missingness.

RESULTS

Moderate correlations existed between occupational scales and FAS and family well-off. Socioeconomic inequalities in poor SRH were found for ISEI, SIOPS and EGP in all regions, independent of FAS and family well-off. Models of imputed data sets did not alter the results. The relationship between SEP and SRH was therefore not biased by high levels of missing values for ISCO.

CONCLUSIONS

ISCO-based indicators of occupational status in cross-national self-administered adolescent health surveys were found to be robust measures of SEP in adolescence. These measure different aspects of SEP independent of FAS and family well-off.

Outcome measures in acute gout: a systematic literature review

OBJECTIVE

Five core domains have been endorsed by Outcome Measures in Rheumatology (OMERACT) for acute gout: pain, joint swelling, joint tenderness, patient global assessment, and activity limitation. We evaluated instruments for these domains according to the OMERACT filter: truth, feasibility, and discrimination.

METHODS

A systematic search strategy for instruments used to measure the acute gout core domains was formulated. For each method, articles were assessed by 2 reviewers to summarize information according to the specific components of the OMERACT filter.

RESULTS

Seventy-seven articles and abstracts met the inclusion criteria. Pain was most frequently reported (76 studies, 20 instruments). The pain instruments used most often were 100 mm visual analog scale (VAS) and 5-point Likert scale. Both methods have high feasibility, face and content validity, and within- and between-group discrimination. Four-point Likert scales assessing index joint swelling and tenderness have been used in numerous acute gout studies; these instruments are feasible, with high face and content validity, and show within- and between-group discrimination. Five-point Patient Global Assessment of Response to Treatment (PGART) scales are feasible and valid, and show within- and between-group discrimination. Measures of activity limitations were infrequently reported, and insufficient data were available to make definite assessments of the instruments for this domain.

CONCLUSION

Many different instruments have been used to assess the acute gout core domains. Pain VAS and 5-point Likert scales, 4-point Likert scales of index joint swelling and tenderness and 5-point PGART instruments meet the criteria for the OMERACT filter.

Bias analysis to improve monitoring an HIV epidemic and its response: approach and application to a survey of female sex workers in Iran

BACKGROUND

We present probabilistic and Bayesian techniques to correct for bias in categorical and numerical measures and empirically apply them to a recent survey of female sex workers (FSW) conducted in Iran.

METHODS

We used bias parameters from a previous validation study to correct estimates of behaviours reported by FSW. Monte-Carlo Sensitivity Analysis and Bayesian bias analysis produced point and simulation intervals (SI).

RESULTS

The apparent and corrected prevalence differed by a minimum of 1% for the number of 'non-condom use sexual acts' (36.8% vs 35.8%) to a maximum of 33% for 'ever associated with a venue to sell sex' (35.5% vs 68.0%). The negative predictive value of the questionnaire for 'history of STI' and 'ever associated with a venue to sell sex' was 36.3% (95% SI 4.2% to 69.1%) and 46.9% (95% SI 6.3% to 79.1%), respectively. Bias-adjusted numerical measures of behaviours increased by 0.1 year for 'age at first sex act for money' to 1.5 for 'number of sexual contacts in last 7 days'.

CONCLUSIONS

The 'true' estimates of most behaviours are considerably higher than those reported and the related SIs are wider than conventional CIs. Our analysis indicates the need for and applicability of bias analysis in surveys, particularly in stigmatised settings.