Testing the External Validity of a Discrete Choice Experiment Method: An Application to Latent Tuberculosis Infection Treatment

Getting it right with discrete choice experiments: Are we hot or cold?

Discrete Choice Experiments (DCEs) are widely employed survey-based methods to assess preferences for healthcare services and products. While they offer an experimental way to represent health-related decisions, the stylized representation of scenarios in DCEs may overlook contextual factors that could influence decision-making. The aim of this paper was to evaluate the predictive validity of preferences elicited through a DCE in decisions likely influenced by a hot-cold empathy gap, and compare it to another commonly used method, a direct-elicitation question. We focused on preferences for pain-relief modalities, especially for an epidural during childbirth - a context where direct-elicitation questions have shown a preference for or intention to have a natural birth (representing the "cold" state), yet individuals often opt for an epidural during labor (representing the "hot" state). Leveraging a unique dataset collected from 248 individuals, we incorporated both the stated preferences collected through a survey administered upon hospital admission for childbirth and the actual pain-relief modality usage data documented in medical records. The DCE allowed for the evaluation of scenarios outside of those expected by respondents to simulate decision-making during childbirth. When we compared the predicted epidural use with the actual epidural use during labor, we observed a choice concordance of 71-60%, depending on the model specification. The concordance rate between the predicted and actual choices increased to 77-76% when accounting for the initial use of other ineffective modalities. In contrast, the direct-elicitation choices, relying solely on respondents' baseline expectations, yielded a lower concordance rate of 58% with actual epidural use. These findings highlight the flexibility of the DCE method in simulating complex decision contexts, including those involving hot-cold empathy gaps. The DCE proves valuable in assessing nuanced preferences, providing a more accurate representation of the decision-making processes in healthcare scenarios.

Understanding Australian pharmacy degree holders' job preferences through the lens of motivation-hygiene theory

Increasing the contribution of pharmacists to primary care has been long discussed, particularly in the context of health workforce shortages and the push to better integrate all providers across primary care. This study examines the employment preferences of Australian pharmacy degree holders (PDHs) elicited through a discrete choice experiment (DCE), to better understand the drivers of current labour force choices. A labelled DCE was developed incorporating the six employment sectors: hospital pharmacy, community pharmacy, primary healthcare settings, pharmaceutical industry, government/academia, and non-pharmacy-related sector. Each alternative was described by five attributes using Herzberg's Two Factor Theory as a conceptual framework. They include motivators - role and career opportunities, and hygiene factors-flexible work schedule, geographic location, and salary. Unforced choice data were analysed using conditional logit and mixed logit models. Based on a sample of 678 PDHs in Australia, our findings indicated pharmaceutical industry is the least preferred sector, followed by non-pharmacy-related sector. Motivators in the form of role and career opportunities are the most important attributes in hospital pharmacy while hygiene factors - geographic location and salary significantly drive the choice of community pharmacy and primary care settings. We provided evidence of a willingness to adopt expanded roles in community pharmacy. This unique interpretation of the key drivers of employment preference in light of motivators and hygiene factors provides policy makers with important information when designing policies to attract and retain PDHs across employment sectors.

Preferences of people living with HIV for features of tuberculosis preventive treatment regimens - a discrete choice experiment

Background

Tuberculosis (TB) preventive treatment (TPT) is recommended for people living with HIV (PLHIV) in high TB burden settings. While 6 months of daily isoniazid remains widely used, shorter regimens are now available. However, little is known about preferences of PLHIV for key features of TPT regimens.

Methods

We conducted a discrete choice experiment among adult PLHIV engaged in care at an urban HIV clinic in Kampala, Uganda. In nine random choice tasks, participants chose between two hypothetical TPT regimens with different features (pills per dose, frequency, duration, need for adjusted antiretroviral therapy [ART] dosage and side effects). We analyzed preferences using hierarchical Bayesian estimation, latent class analysis, and willingness-to-trade simulations.

Results

Of 400 PLHIV, 392 (median age 44, 72% female, 91% TPT-experienced) had high quality choice task responses. Pills per dose was the most important attribute (relative importance 32.4%, 95% confidence interval [CI] 31.6 - 33.2), followed by frequency (20.5% [95% CI 19.7 - 21.3]), duration (19.5% [95% CI 18.6 - 20.5]), and need for ART dosage adjustment (18.2% [95% CI 17.2 - 19.2]). Latent class analysis identified three preference groups: one prioritized less frequent, weekly dosing (N=222; 57%); another was averse to ART dosage adjustment (N=107; 27%); and the last prioritized short and tolerable regimens (N=63; 16%). All groups highly valued fewer pills per dose. Participants were willing to accept a regimen of 2.8 months' additional duration [95% CI: 2.4 - 3.2] to reduce pills per dose from five to one, 3.6 [95% CI 2.4 - 4.8] months for weekly rather than daily dosing, and 2.2 [95% CI 1.3 - 3.0] months to avoid ART dosage adjustment.

Conclusions

To align with preferences of PLHIV, decision-makers should prioritize the development and implementation of TPT regimens with fewer pills, less frequent dosing, and no need for ART dosage adjustment, rather than focus primarily on duration of treatment.

Can healthcare choice be predicted using stated preference data? The role of model complexity in a discrete choice experiment about colorectal cancer screening

INTRODUCTION

The validity of discrete choice experiments (DCEs) is crucial to its usage in healthcare decision-making, but there is only a limited number of health contexts in which external validity is demonstrated. This study aims to assess the internal and external validity of the DCE in the context of colorectal cancer (CRC) screening, and gather insights into the discrepancy between stated and revealed preferences.

METHODS

Stated and revealed preferences were elicited on an individual level from Dutch residents eligible for CRC screening in a DCE and a field experiment, respectively (N = 568). To identify the determinants of CRC screening participation and their relative importance, five random utility maximisation models that varied in complexity were used. We assessed the accuracy with which the models based on stated preferences predict individual-level screening choice in a holdout task (internal validity) and in the actual screening choice (external validity). Insights into the discrepancy between stated and revealed preferences were gathered by comparing groups of respondents.

RESULTS

Our findings show high internal and external validity. Choices could be accurately predicted for 95% of the respondents in the holdout task, and 90% in the actual screening choice. When scale and preference heterogeneity were taken into account model fit improved; individual-level prediction accuracy slightly increased for the holdout task but not for the actual screening choice. Respondents for whom stated preferences matched revealed preferences were generally in better health and found the GP's support for their screening decision more important.

DISCUSSION

Evidence was found that revealed preferences can be predicted accurately on an individual level. Incorporating heterogeneity improved internal validity but not external validity. Differences between stated and revealed preferences can be attributed to respondents' health and the support of their GP. We suggest researchers to continue investigating the internal and external validity of discrete choice experiments, and the role of model complexity.

Preferences of Older People With a Life-Limiting Illness: A Discrete Choice Experiment

CONTEXT

There is limited evidence about which elements and characteristics of palliative care service provision improve the experiences of older people living with life-limiting illness.

OBJECTIVES

To evaluate older patients' (≥65 years) preferences for elements of services and supports and to explore relationships between patient characteristics and the patterns of preferences.

METHODS

A cross-sectional survey undertaken in Ireland and England using a Discrete Choice Experiment with people accessing specialist palliative care services. A random-effects probit model was used to estimate patient preferences.

RESULTS

Of the 77 patients were interviewed, 51 participated in the Discrete Choice Experiment component of the interview (response rate = 66%). Participants prioritized support that minimized unpaid caregiver burden (P < 0.001). They also preferred ease of access to services including out-of-hours access (P < 0.001) and free care at home (P < 0.001). Quality of life was prioritized over quantity of life (<0.001).

CONCLUSION

People living with a life-limiting illness value care that focuses on quality of life, ensures barrier-free access to services and provides sufficient support for relatives. In the context of limited resources and growing demand for care, this study provides evidence about the service elements palliative care delivery models should prioritize and evaluate.

Strategies to enhance the effects of pictorial warnings for cigarettes: results from a discrete choice experiment

AIMS

To measure the effects of changing attributes of pictorial health warning labels (HWLs) on cigarette packs in a country that has already implemented pictorial HWLs.

MEASURES

For each choice set, participants were presented with two cigarette packs and asked the following three questions: (1) 'If only these two cigarette packs were available, which would you buy?'; (2) 'Each of these two packs has warnings on the front and back about the health effects of smoking. Which of these warnings best informs you about the dangers from smoking?'; and (3) 'Which warning most makes you think about quitting smoking?'. As recommended for best practices in discrete choice experiments, each of these questions was followed by an 'opt-out' question for participants to indicate whether they really believed there was a difference between the options presented (i.e. 'Would you really choose one of them?'; 'Do you really think that either of these warnings informs you about dangers from smoking?'; or 'Do you really think that either of these warnings would make you think about quitting smoking?', respectively). Each choice set could be viewed for as long as the participant wished. For each choice question (i.e. willingness to buy, informative, motivating to quit), the pack chosen was coded as 1 and the other pack as 0, with both packs being given a value of 0 if the participant 'opted out'.

DESIGN

A within-subject discrete choice experiment that involved systematic manipulation of pictorial HWL size [75 versus 30% (current policy)]; inclusion of imagery on the back of the pack [versus none (current policy)]; and color formatting [black on yellow versus yellow on black (current policy)].

SETTING

Mexico, on-line panel.

PARTICIPANTS

Adult smokers (n = 705).

MEASUREMENTS

For each choice set, participants selected one pack as having the most informative HWL about smoking harms, the one that makes them think the most about quitting and the one they were most willing to buy. We assessed the independent and interactive effects of HWL attributes on choices.

FINDINGS

Larger HWL size on the pack front (75 versus 30%) and inclusion of a pictorial image on the pack back were both independently associated with lower willingness to buy a pack [b = -0.228, standard error (SE) = 0.023 and -0.089, SE = 0.016, respectively] and greater perception of an HWL as informative (b = 0.214, SE = 0.022, and 0.191, SE = 0.017, respectively) and motivating to quit (b = 0.251, SE = 0.023 and 0.194, SE = 0.017, respectively). HWL with black text and yellow background were perceived as less informative (b = -0.037, SE = 0.016) and less motivating to quit (b = -0.032, SE = 0.015) compared with yellow text on a black background.

CONCLUSIONS

Among adult Mexican smokers, pictorial health warning labels on cigarette packages that are larger or cover both sides of the pack appear more effective at lowering purchase intentions and increasing risk perceptions and motivation to quit than smaller health warning labels or health warning labels with imagery only on the pack front.

HIV Pre-exposure Prophylaxis Implant Stated Preferences and Priorities: Results of a Discrete Choice Experiment Among Women and Adolescent Girls in Gauteng Province, South Africa

For adolescent girls (AG) and young women (YW), adherence barriers may limit the effectiveness of daily oral HIV pre-exposure prophylaxis (PrEP). Due to its low-burden and long-lasting product attributes, PrEP implants could remove some of the critical adherence barriers of oral PrEP products for individuals at risk of HIV. To explore stated preferences for a long-acting PrEP implant, we conducted a quantitative survey and discrete choice experiment with AG (ages 15-17), YW (18-34), and female sex workers (FSW; ≥ 18) in Gauteng Province, South Africa. We completed 600 quantitative surveys across the three subgroups of women. Respondents stated preference for an implant that provided longer HIV protection (24 months versus 6 months) and required a single insertion. They stated that they preferred a biodegradable implant that could be removed within 1 month of insertion. Respondents had no preference for a particular insertion location. Overall, 78% of respondents said they would be likely (33%) or very likely (45%) to use a PrEP implant were one available, with the majority (82%) stating preference for a product that would provide dual protection against HIV and unintended pregnancies. To reduce their risk of HIV, AG, YW, and FSW in our survey reported a strong willingness to use long-acting, highly-effective, dissolvable PrEP implants.

Location, location, location: a discrete choice experiment to inform COVID-19 vaccination programme delivery in the UK

BACKGROUND

Large-scale vaccination is fundamental to combatting COVID-19. In March 2021, the UK's vaccination programme had delivered vaccines to large proportions of older and more vulnerable population groups; however, there was concern that uptake would be lower among young people. This research was designed to elicit the preferences of 18-29-year-olds regarding key delivery characteristics and assess the influence of these on intentions to get vaccinated, to inform planning for this cohort.

METHODS

From 25 March to 2 April 2021, an online sample of 2012 UK adults aged 18-29 years participated in a Discrete Choice Experiment. Participants made six choices, each involving two SMS invitations to book a vaccination appointment and an opt-out. Invitations had four attributes (1 × 5 levels, 3 × 3 levels): delivery mode, appointment timing, proximity, and sender. These were systematically varied according to a d-optimal design. Responses were analysed using a mixed logit model.

RESULTS

The main effects logit model revealed a large alternative-specific constant (β = 1.385, SE = 0.067, p < 0.001), indicating a strong preference for 'opting in' to appointment invitations. Pharmacies were dispreferred to the local vaccination centre (β = - 0.256, SE = 0.072, p < 0.001), appointments in locations that were 30-45 min travel time from one's premises were dispreferred to locations that were less than 15 min away (β = - 0.408, SE = 0.054, p < 0.001), and, compared to invitations from the NHS, SMSs forwarded by 'a friend' were dispreferred (β = - 0.615, SE = 0.056, p < 0.001) but invitations from the General Practitioner were preferred (β = 0.105, SE = 0.048, p = 0.028).

CONCLUSIONS

The results indicated that the existing configuration of the UK's vaccination programme was well-placed to deliver vaccines to 18-29-year-olds; however, some adjustments might enhance acceptance. Local pharmacies were not preferred; long travel times were a disincentive but close proximity (0-15 min from one's premises) was not necessary; and either the 'NHS' or 'Your GP' would serve as adequate invitation sources. This research informed COVID-19 policy in the UK, and contributes to a wider body of Discrete Choice Experiment evidence on citizens' preferences, requirements and predicted behaviours regarding COVID-19.

Examining Ways to Improve Weight Control Programs' Population Reach and Representativeness: A Discrete Choice Experiment of Financial Incentives

BACKGROUND

Both theoretical and empirical evidence supports the potential of modest financial incentives to increase the reach of evidence-based weight control programs. However, few studies exist that examine the best incentive design for achieving the highest reach and representativeness at the lowest cost and whether or not incentive designs may be valued differentially by subgroups that experience obesity-related health disparities.

METHODS

A discrete choice experiment was conducted (n = 1232 participants; over 90% of them were overweight/obese) to collect stated preference towards different financial incentive attributes, including reward amount, program location, reward contingency, and payment form and frequency. Mixed logit and conditional logit models were used to determine overall and subgroup preference ranking of attributes. Using the National Health and Nutrition Examination Survey data sample weights and the estimated models, we predicted US nationally representative participation rates by subgroups and examined the effect of offering more than one incentive design. External validity was checked by using a completed cluster randomized control trial.

RESULTS

There were significant subgroup differences in preference toward incentive attributes. There was also a sizable negative response to larger incentive amounts among African Americans, suggesting that higher amounts would reduce participation from this population. We also find that offering participants a menu of incentive designs to choose from would increase reach more than offering higher reward amounts.

CONCLUSIONS

We confirmed the existence of preference heterogeneity and the importance of subgroup-targeted incentive designs in any evidence-based weight control program to maximize population reach and reduce health disparities.

Effects of tobacco product type and characteristics on appeal and perceived harm: Results from a discrete choice experiment among Guatemalan adolescents

Guatemala is one of the few countries where both heated tobacco products (HTPs) and electronic cigarettes (e-cigarettes) remain unregulated. We used a discrete choice experiment (DCE) administered to 2038 high school students to assess how tobacco product attributes influence their appeal among Guatemalan adolescents. Participants were randomly assigned to evaluate 4 of 32 contrasting sets, each containing 3 packs (1 of each product type). Experimental manipulations included: product type, brand, nicotine content and flavor. Participants then indicated which product they were most and least interested in trying and would be most and least harmful to their health. Conditional logistic regression models were used to assess the impact of product characteristics on choice. Product type accounted for almost 90% of variation in choices. Respondents were less interested in trying HTPs (B = -0.93; p < 0.001) and viewed them as more harmful (B = 2.77; p < 0.001) compared to cigarettes. They were more interested in trying e-cigarettes (B = 1.22; p < 0.001), which were also perceived as less harmful (B = -1.47; p < 0.001) compared to cigarettes. Products without nicotine were of more interest for trying (B = 0.14; p < 0.001) and perceived as more harmful (B = 0.20; p < 0.001) than those with. Students were more interested in trying a flavor compared to regular tobacco and among the flavors, berry was the highest rated one (B = 0.28; p < 0.001). Finally, in this country with weak tobacco control, e-cigarettes appear to be more appealing and perceived as less harmful than HTPs and cigarettes. Packaging and flavoring regulations are urgently needed on these products as they are a marketing strategy targeting adolescents.

Parents' preferences for nursery care when children are unwell: a discrete choice experiment

BACKGROUND

Pre-school children's daycare is associated with increased incidence of respiratory and diarrhoeal illnesses. While the incidence might be reduced if all unwell children were kept at home, parental employment pressures make this difficult when children are marginally unwell.

METHODS

A discrete choice experiment (DCE) was conducted to identify what aspects of daycare policy and provision would affect parents' decisions to keep marginally unwell children home. Prior qualitative research informed parameter choice. The DCE was accompanied by a best-worst scaling task examining preferences for four modifiable aspects of care: swapping unused daycare sessions, reimbursing unused sessions, daycare paracetamol policy and presence of a 'quiet room'.

RESULTS

Paracetamol guidelines and the presence of a quiet room had the strongest predicted influence on parents' decision-making. Conditional on assumptions about the set-up of the daycare, introducing a 'no paracetamol' policy would result in a fall from 62 to 25% in mean predicted probabilities of a parent sending a marginally unwell child to nursery, while introducing a quiet room would increase the mean probability from 34 to 53%.

CONCLUSIONS

Daycare policy, particularly the use of paracetamol prior to attendance, could impact parents' decisions to send unwell children to daycare, potentially influencing the transmission of children's infectious illness.

Assessing cigarette packaging and labelling policy effects on early adolescents: results from a discrete choice experiment

INTRODUCTION

Cigarette packaging is a primary channel for tobacco advertising, particularly in countries where traditional channels are restricted. The current study evaluated the independent and interactive effects of cigarette packaging and health warning label (HWL) characteristics on perceived appeal of cigarette brands for early adolescents in Mexico.

METHODS

A discrete choice experiment (DCE) was conducted with early adolescents, aged 12-14 years (n=4251). The DCE involved a 3×25 design with six attributes: brand (Marlboro, Pall Mall, Camel), tobacco flavour (regular, menthol), flavour capsule (none, 1 or 2 capsules), presence of descriptive terms, branding (vs plain packaging), HWL size (30%, 75%) and HWL content (emphysema vs mouth cancer). Participants viewed eight sets of three cigarette packs and selected a pack in each set that: (1) is most/least attractive, (2) they are most/least interested in trying or (3) is most/least harmful, with a no difference option.

RESULTS

Participants perceived packs as less attractive, less interesting to try and more harmful if they had plain packaging or had larger HWLs, with the effect being most pronounced when plain packaging is combined with larger HWLs. For attractiveness, plain packaging had the biggest influence on choice (43%), followed by HWL size (19%). Interest in trying was most influenced by brand name (34%), followed by plain packaging (29%). Perceived harm was most influenced by brand name (30%), followed by HWL size (29%).

CONCLUSION

Increasing the size of HWLs and implementing plain packaging appear to reduce the appeal of cigarettes to early adolescents. Countries should adopt these policies to minimise the impact of tobacco marketing.

Public Attitudes on Lung Cancer Screening and Radiation Risk: A Best-Worst Experiment

OBJECTIVES

To measure Australian population preferences for lung cancer screening and to explore whether these preferences are related to respondent characteristics and lung cancer risk.

METHODS

An online ranking task was administered to a sample of 521 Australians between the ages of 50 and 80 with a history of cigarette smoking. Choice sets contained 2 alternative lungs screens and an opt-out, and respondents were asked to rank the 3 options. Both conditional logit and mixed logit analyses were conducted exploring both the forced choice between the 2 screens and identifying the types of respondent most likely to opt out of any screening. For this, respondent 6-year lung cancer risk was estimated and used as a covariate.

RESULTS

Respondents valued tests that involved breath or blood tests in addition to computerized tomography (CT), locations that were close to home, receiving results quickly, and minimizing radiation from the CT scan. Willingness to pay differed between relatively higher and lower risk individuals; higher risk individuals placed greater emphasis on convenience, result timeliness, and radiation. Respondent characteristics that predicted opting out of any screening included being male, fewer years of smoking, and not having a previous cancer diagnosis. Lung cancer risk did not influence the likelihood of opting out.

CONCLUSIONS

Uptake of lung cancer screening is likely to be changeable if different modalities of screening are provided, with effects likely differing across population subgroups.

Are Healthcare Choices Predictable? The Impact of Discrete Choice Experiment Designs and Models

BACKGROUND

Lack of evidence about the external validity of discrete choice experiments (DCEs) is one of the barriers that inhibit greater use of DCEs in healthcare decision making.

OBJECTIVES

To determine whether the number of alternatives in a DCE choice task should reflect the actual decision context, and how complex the choice model needs to be to be able to predict real-world healthcare choices.

METHODS

Six DCEs were used, which varied in (1) medical condition (involving choices for influenza vaccination or colorectal cancer screening) and (2) the number of alternatives per choice task. For each medical condition, 1200 respondents were randomized to one of the DCE formats. The data were analyzed in a systematic way using random-utility-maximization choice processes.

RESULTS

Irrespective of the number of alternatives per choice task, the choice for influenza vaccination and colorectal cancer screening was correctly predicted by DCE at an aggregate level, if scale and preference heterogeneity were taken into account. At an individual level, 3 alternatives per choice task and the use of a heteroskedastic error component model plus observed preference heterogeneity seemed to be most promising (correctly predicting >93% of choices).

CONCLUSIONS

Our study shows that DCEs are able to predict choices-mimicking real-world decisions-if at least scale and preference heterogeneity are taken into account. Patient characteristics (eg, numeracy, decision-making style, and general attitude for and experience with the health intervention) seem to play a crucial role. Further research is needed to determine whether this result remains in other contexts.

Preferences of Patients and At-risk Individuals for Preventive Approaches to Rheumatoid Arthritis

Effective treatments for rheumatoid arthritis (RA) are available and can lead to remission for some patients, but most patients remain on potentially toxic and expensive medications in the long term. Interest is increasingly turning to the disease phases preceding the development of RA that represent opportunities for preventive interventions. At-risk target populations include individuals with genetic and environmental risk factors, those who have developed systemic autoimmunity, and those who have developed clinically suspect symptoms (eg, arthralgias without synovitis, or an early arthritis). Ongoing prospective studies will inform the development of increasingly accurate predictive tools to identify individuals at risk of developing RA. Furthermore, a range of preventive approaches has been suggested, including lifestyle modification (eg, smoking cessation) and pharmacologic interventions (eg, hydroxychloroquine, methotrexate, abatacept, rituximab) that are currently the subject of randomized controlled trials. As prediction and prevention of RA evolve, it is increasingly likely that individuals at risk (including asymptomatic individuals) may be faced with complex decisions about whether to accept assessment of their risk status or to take a preventive intervention associated with risk of serious adverse events and uncertain benefit. Acceptance of preventive medication in other contexts can be low. For example, <25% of women at high risk of breast cancer are willing to take preventive hormonal treatments. Actual uptake is lower still. Patients' beliefs and preferences predict treatment uptake and adherence. Before the dream of preventing RA can become reality, health care providers need to understand the perspectives of individuals in the target population and to identify barriers and facilitators for this approach. This commentary reviews what is currently known about the perspectives of patients and individuals at risk about predictive and preventive approaches for RA and identifies gaps to be addressed to inform the development of efficient preventive strategies.

Quantifying Family Spillover Effects in Economic Evaluations: Measurement and Valuation of Informal Care Time

Spillover effects on the welfare of family members may refer to caregiver health effects, informal care time costs, or both. This review focuses on methods that have been used to measure and value informal care time and makes suggestions for their appropriate use in cost-of-illness and cost-effectiveness analyses. It highlights the importance of methods to value informal care time that are independent of caregiver health effects in order to minimize double counting of spillover effects. Although the concept of including caregiver time costs in economic evaluations is not new, relatively few societal perspective cost-effectiveness analyses have included informal care, with the exception of dementia. This is due in part to challenges in measuring and valuing time costs. Analysts can collect information on time spent in informal care or can assess its impact in displacing other time use, notably time in paid employment. A key challenge is to ensure appropriate comparison groups that do not require informal care to be able to correctly estimate attributable informal care time or foregone market work. To value informal care time, analysts can use estimates of hourly earnings in either opportunity cost or replacement cost approaches. Researchers have used widely varying estimates of hourly earnings. Alternatively, stated-preference methods (i.e. contingent valuation, conjoint analysis) can be used to value the effect of informal care on utility, but this can entail double counting with health effects. Lack of consensus and standardization of methods makes it difficult to compare estimates of informal care costs.

How well do discrete choice experiments predict health choices? A systematic review and meta-analysis of external validity

Discrete choice experiments (DCEs) are economic tools that elicit the stated preferences of respondents. Because of their increasing importance in informing the design of health products and services, it is critical to understand the extent to which DCEs give reliable predictions outside of the experimental context. We systematically reviewed the literature of published DCE studies comparing predictions to choices made in reality; we extracted individual-level data to estimate a bivariate mixed-effects model of pooled sensitivity and specificity. Eight studies met the inclusion criteria, and six of these gave sufficient data for inclusion in a meta-analysis. Pooled sensitivity and specificity estimates were 88% (95% CI 81, 92%) and 34% (95% CI 23, 46%), respectively, and the area under the SROC curve (AUC) was 0.60 (95% CI 0.55, 0.64). Results indicate that DCEs can produce reasonable predictions of health-related behaviors. There is a great need for future research on the external validity of DCEs, particularly empirical studies assessing predicted and revealed preferences of a representative sample of participants.