Assessing the comparative feasibility, acceptability and equivalence of videoconference interviews and face-to-face interviews using the time trade-off technique

A Methodological Study to Compare Alternative Modes of Administration With Value EQ-5D Using Preference-Elicitation Techniques

OBJECTIVES

Time trade-off (TTO) and discrete choice experiment (DCE) preference-elicitation techniques can be administered using face-to-face interviews (F2F), unassisted online (UO) surveys, or remote-assisted (RA) interviews. The objective of this study was to explore how the mode of administration affects the quality and reliability of preference-elicitation data.

METHODS

EQ-5D-5L health states were valued using composite TTO (cTTO) and DCE approaches by the UK general population. Participants were allocated to 1 of 2 study groups. Group A completed both F2F and UO surveys (n = 271), and group B completed both RA and UO surveys (n = 223). The feasibility of survey completion and the reliability and face-validity of data collected were compared across all modes of administration.

RESULTS

Fewer participants reported receiving sufficient guidance on the cTTO tasks during the UO survey compared with the 2 assisted modes. Participants across all modes typically reported receiving sufficient guidance on the DCE tasks. cTTO data were less reliable from the UO survey compared with both assisted modes, but there were no differences in DCE data reliability. cTTO data from all modes demonstrated face-validity; however, the UO survey produced higher utilities for moderate and severe health states than both assisted modes. Both F2F and RA modes provided comparably reliable data.

CONCLUSIONS

The reliability of DCE data is not affected by the mode of administration. Interviewer-assisted modes of administration (F2F or RA) yield more reliable cTTO data than unassisted surveys. Both F2F and RA surveys produced similar-quality data.

Testing "Pits" Time Trade-Off: Can Data Quality be Improved by Removing Death From Valuation of Health States?

OBJECTIVES

Composite time trade-off (cTTO) often exhibits issues such as a value clustering at -1, potentially due to suboptimal valuation task design. We developed a novel time trade-off which uses the "pits" state as an anchor state and enables a unified time trade-off procedure for all health states (referred to as "pTTO"). This study aimed to test the pTTO for valuation of EQ-5D-5L health states.

METHODS

A total of 120 members of the general public were invited to value 3 pairs of dominant/dominated EQ-5D-5L states using both cTTO and pTTO and the "pits" state using cTTO. We compared the pTTO with cTTO in terms of feasibility (interviewer-rated task-completing difficulty), acceptability (participant-reported task experience), value distribution, logical consistency, and discriminatory ability (using the standardized response mean, an effect size measure, derived from 3 pairwise comparisons of states with dominant/dominated relationship).

RESULTS

The pTTO demonstrated similar feasibility to cTTO but slightly lower acceptability. The pTTO values exhibited a smoother and more continuous distribution compared with cTTO values. Relative to the cTTO, the pTTO showed higher individual-level logical consistency rate for the pairs comprising severe states (stringent criterion: 53.04% vs 17.39%; less stringent criterion: 82.61% vs 78.26%). The standardized response mean value based on pTTO values were higher than those based on cTTO values for the pair comprising severe states (0.388 vs -0.140).

CONCLUSIONS

The pTTO appeared to be feasible and acceptable to value EQ-5D-5L states. The pTTO values were less clustered, more logically consistent, and discriminative compared with the cTTO values for severe health states.

Behind the Scenes: A Qualitative Investigation of Interviewers' Performance in EQ-5D Valuation Studies

BACKGROUND

The EuroQol Valuation Technology (EQ-VT) protocol is currently employed by the valuation studies of the EQ family of instruments worldwide. To date, all the evidence in support of the quality control (QC) originates from quantitative indicators.

OBJECTIVE

We aimed to explore interviewers' conversational patterns in EQ-VT interviews, beyond quantitative QC indicators, and to provide a preliminary exploration of how the interaction between interviewer and respondent impacts data quality.

METHODS

Two researchers transcribed and independently coded 24 video-recorded interviews from the Italian EQ-5D-5L valuation study, adopting the conversational analysis framework. The analysis identified positive and negative 'patterns' of conversational practice. These were categorized into themes and sub-themes and were used to score a random sample of 42 video-recorded interviews conducted at different time points by seven interviewers.

RESULTS

The conversational analysis identified 20 positive and 14 negative interview patterns, which were grouped into two main themes (i.e., task execution and communication skills). Positive items included appending questions that stimulated respondents' engagement, providing different explanations for an unclear aspect, supporting the participant with useful information for completing the tasks, and increasing the interview's coherence by confirming the respondent answers. Negative patterns included moving forward in the exercise without making sure that the respondent understood, trying to force an answer from the respondent, speaking too fast, and providing incomplete or incorrect explanations of the task. Most interviewers exhibited a moderate increase in positive patterns or a decrease in negative patterns over time. A certain degree of consistency between the quantitative QC results and the qualitative scoring deriving from the interviewer-respondent interaction was observed, with the best and worst performers of the qualitative scoring showing good and bad scores on key QC items, respectively.

CONCLUSIONS

The identified positive and negative patterns may be useful to inform the training material of EQ-VT studies worldwide and complement the existing QC process.

Valuing selected WAItE health states using the Time Trade-Off methodology: findings from an online interviewer-assisted remote survey

PURPOSE

The Weight-Specific Adolescent Instrument for Economic Evaluation (WAItE) is a physical weight-specific patient reported outcome measure for use in adolescence. The purpose of this study was to use the Time Trade-Off (TTO) methodology, administered using an online interviewer-assisted remote survey, to obtain utility values for several health states from the WAItE descriptive system from a sample of the UK adult general population.

METHODS

The adult sample was gathered using a market research company and a sample of local residents. All participants completed the same interviewer-assisted remote survey, which included rating WAItE states of varying impairment using the TTO.

RESULTS

42 adults completed the survey. Utility values were gathered for four health states, ranging from low impairment to the most severe health from the WAItE descriptive system (the Pits state). Consistent orderings of the WAItE health states were observed; the health state with the lowest level of impairment was valued highest and the Pits state was valued lowest. Several respondents (n = 7, 17%) considered the Pits state to be worse than death; however, the mean value of this health state was 0.23.

CONCLUSIONS

The utility value of the Pits state relative to death generated from this study will be used to anchor latent values for WAItE health states generated from a Discrete Choice Experiment onto the 0 = death, 1 = full health Quality Adjusted Life Year (QALY) scale as part of a valuation study for the WAItE in the UK population. This study also provides further evidence that interviewer-assisted digital studies are feasible for collecting TTO data.

Development of a value-based scoring system for the MobQoL-7D: a novel tool for measuring quality-adjusted life years in the context of mobility impairment

PURPOSE

To create a preference-based value set scoring system for the MobQoL-7D outcome measure, and to examine differences in the health state preferences of the general population and individuals with impaired mobility.

METHODS AND MATERIALS

A preference elicitation study was undertaken to ascribe utility weights to all health states (i.e., all unique combination of answers) described by the MobQoL-7D. The elicitation exercise was developed using the Online Elicitation of Personal Utility Functions (OPUF) tool. Two UK sample groups were recruited; firstly a representative general population sample (N = 504), secondly a balanced sample of individuals with impaired mobility (N = 368). Distinct preference-based value sets were developed for each sample. Differences in dimension ranking, weighting, and overall utility values were assessed.

RESULTS

The general population sample considered most health states, especially the more severe states, to be worse than the mobility impaired sample comparatively. Statistically significant differences between the samples were observed in four of the seven MobQoL-7D dimensions.

CONCLUSIONS

This study is the first to provide preference-based value sets for the MobQoL-7D, ready for use in economic evaluations, QALY calculation, and other clinical or research applications. The study demonstrates how the general public and individuals with impaired mobility value health states differently.

Health state utilities associated with weight loss: preferences of people with type 2 diabetes and obesity in Japan

AIMS

Health state utilities associated with weight change are needed for cost-utility analyses (CUAs) examining the value of treatments for type 2 diabetes and obesity. Previous studies have estimated the utility benefits associated with various amounts of weight reduction in the US and Europe, but preferences for weight change in Asian cultures may differ from these published values. The purpose of this study was to estimate utilities associated with reductions in body weight based on preferences of individuals with type 2 diabetes and obesity in Japan.

METHODS

Health state vignettes represented type 2 diabetes with respondents' own current weight and weight reductions of 2.5%, 5%, 7.5%, 10%, 12.5%, 15%, and 20%. Utilities were elicited in time trade-off interviews with a sample of respondents in Japan with type 2 diabetes and body mass index (BMI) ≥25 kg/m2 (the cutoff for obesity in Japan).

RESULTS

Analyses were conducted with data from 138 respondents (84.8% male; mean age = 58.0 years; mean BMI = 29.4 kg/m2) from all eight regions of Japan. Utility gains gradually increased with rising percentage of weight reductions ranging from 2.5% to 15%. Weight reductions of 2.5% to 15% resulted in utility increases of 0.013 to 0.048. The health state representing a 20% weight reduction yielded a wide range of preferences (mean utility increase of 0.044). Equations are recommended for estimating utility change based on any percentage of weight reduction (up to 20%) in Japanese people with type 2 diabetes and obesity.

LIMITATIONS

This study was conducted in a sample with limited representation of patients with BMI >35 kg/m2 (n = 13) and relatively few women (n = 21).

CONCLUSION

Results may be used to provide inputs for CUAs examining the value of treatments that are associated with weight loss in patients with type 2 diabetes and obesity in Japan.

UK Valuation of EQ-5D-5L, a Generic Measure of Health-Related Quality of Life: A Study Protocol

OBJECTIVES

A high-quality and widely accepted UK EQ-5D-5L value set is urgently required to enable the latest version of EQ-5D scored using recent UK public preferences to inform policy including health technology assessments submitted to the National Institute for Health and Care Excellence. This article outlines the study protocol for the generation of a new EQ-5D-5L UK value set.

METHODS

Twelve hundred interviews will be undertaken using the composite time trade-off elicitation technique for 102 health states (86 from the international EQ-5D-5L valuation protocol, plus 16 with best predictive performance in an extended design used in the Native American EQ-5D-5L valuation). The sample will be UK adults (age ≥18 years) proportionately representative across England, Wales, Scotland, and Northern Ireland, representative for age, sex, ethnicity, and socioeconomic group, with inclusion of participants with/without health problems. Participants will choose to be interviewed via videoconference (by Zoom) or in-person in a central venue. Data quality will be rigorously assessed.

RESULTS

The value set will be generated using tobit random effects and heteroscedastic tobit models (with censoring at -1) using all data, excluding time trade-off values highlighted by participants as ones they would reconsider and data from interviewers failing protocol compliance. Quality and acceptance will be achieved by public involvement, regular Steering Group meetings, independent assessment of data quality at 4 time points, and final endorsement of data and analyses.

CONCLUSION

This study will produce a UK value set for the EQ-5D-5L for use in prospective and retrospective data sets containing EQ-5D-5L data.

Estimating health related quality of life effects in vitiligo. Mapping EQ-5D-5 L utilities from vitiligo specific scales: VNS, VitiQoL and re-pigmentation measures using data from the HI-Light trial

BACKGROUND

Vitiligo is reported to affect 2% of the world's population and has a significant impact on health related quality of life (HRQoL). The relationship between HRQoL and clinical outcomes used in vitiligo require further examination. Mapping condition specific measures of HRQoL: vitiligo specific quality of life instrument (VitiQoL), vitiligo noticeability scale (VNS) and vitiligo re-pigmentation scores (RPS) to the EQ-5D have not yet been reported.

METHODS

Data collected from a randomised clinical trial (HI-Light) in vitiligo was used to develop mapping algorithms for the EQ-5D-5 L and the relationship between HRQoL, clinical outcomes and EQ-5D were investigated. Two EQ-5D-5 L value sets (Van Hout and Alava) using linear and non-linear models were considered. Logistic regression models were used to model the probability of vitiligo noticeability (VNS) in terms of RPS, EQ-5D and VitiQoL scores.

RESULTS

Mapping from RPS appeared to perform better followed by VNS for the Alava crosswalks using polynomial models: Mean observed vs. predicted utilities of 0.9008 (0.005) vs. 0.8984 (0.0004) were observed for RPS. For VNS, mean observed vs. predicted utilities of 0.9008 (0.005) vs. 0.8939 (0.0003) were observed. For VitiQoL, mean observed vs. predicted utilities of 0.9008 (0.005) vs. 0.8912 (0.0002) were observed. For patients with the least re-pigmentation (RPS < 25%), a Total VitiQoL score of about 20 points gives around an 18% chance of vitiligo being no longer or a lot less noticeable.

CONCLUSION

The algorithm based on RPS followed by VNS performed best. The relationship between effects from vitiligo specific HRQoL instruments and clinical RPS was established allowing for plausible clinically relevant differences to be identified, although further work is required in this area.

Valuing the EQ Health and Wellbeing Short Using Time Trade-Off and a Discrete Choice Experiment: A Feasibility Study

OBJECTIVES

The EQ Health and Wellbeing Short (EQ-HWB-S) is a new generic measure that covers health and wellbeing developed for use in economic evaluation in health and social care. The aim was to test the feasibility of using composite time trade-off (cTTO) and a discrete choice experiment (DCE) based on an international protocol to derive utilities for the EQ-HWB-S and to generate a pilot value set.

METHODS

A representative UK general population was recruited. Online videoconference interviews were undertaken where cTTO and DCE tasks were administered using EuroQol Portable Valuation Technology. Quality control (QC) was used to assess interviewers' performance. Data were modeled using Tobit, probit, and hybrid models. Feasibility was assessed based on the distribution of data, participants, and reports of understanding from the interviewer, QC and modeling results.

RESULTS

cTTO and DCE data were available for 520 participants. Demographic characteristics were broadly representative of the UK general population. Interviewers met QC requirements. cTTO values ranged between -1 to 1 with increasing disutility associated with more severe states. Participants understood the tasks and the EQ-HWB-S states; and the interviewers reported high levels of understanding and engagement. The hybrid Tobit heteroscedastic model was selected for the pilot value set with values ranging from -0.384 to 1. Pain, mobility, daily activities, and sad/depressed had the largest disutilities, followed by loneliness, anxiety, exhaustion, control, and cognition in the selected model.

CONCLUSIONS

EQ-HWB-S can be valued using cTTO and DCE. Further methodological work is recommended to develop a valuation protocol specific to the EQ-HWB-S.

Discrete Choice Experiments in Health State Valuation: A Systematic Review of Progress and New Trends

BACKGROUND

Discrete choice experiments (DCEs) are increasingly used in health state valuation studies.

OBJECTIVE

This systematic review updates the progress and new findings of DCE studies in the health state valuation, covering the period since the review of June 2018 to November 2022. The review reports the methods that are currently being used in DCE studies to value health and study design characteristics, and, for the first time, reviews DCE health state valuation studies published in the Chinese language.

METHODS

English language databases PubMed and Cochrane, and Chinese language databases Wanfang and CNKI were searched using the self-developed search terms. Health state valuation or methodology study papers were included if the study used DCE data to generate a value set for a preference-based measure. Key information extracted included DCE study design strategies applied, methods for anchoring the latent coefficient on to a 0-1 QALY scale and data analysis methods.

RESULTS

Sixty-five studies were included; one Chinese language publication and 64 English language publications. The number of health state valuation studies using DCE has rapidly increased in recent years and these have been conducted in more countries than prior to 2018. Wide usage of DCE with duration attributes, D-efficient design and models accounting for heterogeneity has continued in recent years. Although more methodological consensus has been found than in studies conducted prior to 2018, this consensus may be driven by valuation studies for common measures with an international protocol (the 'model' valuation research). Valuing long measures with well-being attributes attracted attention and more realistic design strategies (e.g., inconstant time preference, efficient design and implausible states design) were identified. However, more qualitative and quantitative methodology study is still necessary to evaluate the effect of those new methods.

CONCLUSIONS

The use of DCEs in health state valuation continues to grow dramatically and the methodology progress makes the method more reliable and pragmatic. However, study design is driven by international protocols and method selection is not always justified. There is no gold standard for DCE design, presentation format or anchoring method. More qualitative and quantitative methodology study is recommended to evaluate the effect of new methods before researchers make methodology decisions.

Randomised comparison of online interviews versus face-to-face interviews to value health states

INTRODUCTION

Health state valuation studies using composite time trade-off (cTTO) interviews have historically been conducted face-to-face. The COVID-19 pandemic forced disruptive innovation meaning a number of valuation studies conducted interviews via videoconference. These studies found online interviews feasible and acceptable; however, studies were not constructed to test the impact of online versus face-to-face interviews. This study builds on its sister study from the UK and aims to assess the acceptability and equivalence of in person face-to-face interviews with online interviews on cTTO valuation outcomes and on data quality.

METHODS

Participants were recruited into a randomised equivalence study via an external research company. Consenting participants were randomly allocated to complete a cTTO interview face-to-face or online, using the same 10 EQ-5D-5L health states. Mean and distribution of the cTTO values, participant understanding, data quality, demographic characteristics, participant preference, participant engagement and participant feedback were all compared across interview mode. Statistical equivalence for cTTO values for each state was tested using two one-sided t-tests by mode. Finally, regression analysis was completed to assess the impacts of interview mode on cTTO value while controlling for demographic characteristics of the participants.

RESULTS

Mean cTTO values were shown to be equivalent for mild health states and showed no significant difference for serious health states. The proportion of individuals who expressed an interest in the study but declined to arrange an interview after finding out their randomisation was significantly higher for the face-to-face (21.6%) than the online group (1.8%). No significant difference was found between groups for participant engagement, understanding or feedback nor for any indicators of data quality.

CONCLUSION

Administrating interviews face to face or online did not appear to have a statistically significant impact on mean cTTO values. Offering both online and face-to-face interviews routinely allows all participants to select the most convenient option.

A Value Set for the EQ-5D-Y-3L in the Netherlands

BACKGROUND AND OBJECTIVE

There is increasing interest in preference-accompanied measures of health for paediatric populations. The child-friendly EQ-5D version, EQ-5D-Y-3L, is one such instrument, but the lack of a Dutch value set prevents its use in economic evaluations of healthcare interventions in the Netherlands. This study aims at covering this gap by collecting preferences using a standardised protocol for deriving EQ-5D-Y-3L value sets.

METHODS

Composite time trade-off data were collected using videoconferencing interviews, with each respondent completing ten composite time trade-off tasks. Discrete choice experiment data were collected using an online survey, with respondents each completing 15 paired comparisons. Respondents completed these tasks considering what they prefer for a hypothetical 10-year-old child. Discrete choice experiment data were analysed using a ten-parameter mixed-logit model and anchored to the quality-adjusted life-year scale using the mean observed composite time trade-off values.

RESULTS

The study collected preferences for 197 respondents using composite time trade-off and for 959 respondents using a discrete choice experiment. The discrete choice experiment sample was representative for the Dutch population in terms of age and sex. The level 3 weight for pain/discomfort was the largest, followed by feeling worried, sad or unhappy, usual activities, mobility and self-care. Health state values ranged between -0.218 and 1.

CONCLUSIONS

This study generated a Dutch value set for the EQ-5D-Y-3L, which can be used for the computation of quality-adjusted life-years for economic evaluations of healthcare interventions in paediatric populations.

In a Child's Shoes: Composite Time Trade-Off Valuations for EQ-5D-Y-3L with Different Proxy Perspectives

BACKGROUND AND OBJECTIVES

EQ-5D-Y-3L health states are commonly valued by asking adults to complete stated preference tasks, 'given their views about a 10-year-old child' (hereafter referred to as proxy 1). The use of this perspective has been a source of debate. In this paper, we investigated an alternative proxy perspective: i.e. adults considered what they think a 10-year old-child would decide for itself (hereafter, proxy 2 (substitute)]. Our main objective was to explore how the outcomes, dispersion and response patterns of a composite time trade-off valuation differ between proxy 1 and proxy 2.

METHODS

A team of four trained interviewers completed 402 composite time trade-off interviews following the EQ-5D-Y-3L protocol. Respondents were randomly allocated to value health states in either the proxy 1 or proxy 2 (substitute) perspective. Each respondent valued ten health states with the perspective they were assigned to, as well as one health state with the alternative perspective (33333).

RESULTS

The use of different proxy perspectives yielded differences in EQ-5D-Y-3L valuation. For states in which children had considerable pain and were very worried, sad or unhappy, respondents' valuations were lower in proxy 1 than in proxy 2 (substitute) perspectives, by about 0.2. Within-subject variation across health states was lower for proxy 2 (substitute) than proxy 1 perspectives. Analyses of response patterns suggest that data for proxy 2 (substitute) perspectives were less clustered.

CONCLUSIONS

There are systematic differences between composite time trade-off responses given by adults deciding for children and adults considering what children would want for themselves. In addition to warranting further qualitative exploration, such differences contribute to the ongoing normative discussion surrounding the source and perspective used for valuation of child and adolescent health.