Quantitative decision analysis: a work in progress

Good decision making requires good communication

The methods used for regulatory decisions must facilitate three kinds of communication: (i) with individual experts who must translate their knowledge into usable form; (ii) among the experts whose pooled knowledge informs those choices; and (iii) between regulators and those affected by their choices. Decision-making methods vary in their reliance on expert judgement and computational methods and, hence, in their ability to meet the goals of sound decision making: breadth, depth, precision, neutrality, evaluability and transparency. An approach developed by the US FDA, the Benefit-Risk Framework, integrates judgement and computation, cognizant of their strengths and weaknesses. Its application both requires and facilitates good communication about risks and benefits.

Quantitative benefit-risk assessment using only qualitative information on utilities

BACKGROUND

Utilities of pertinent clinical outcomes are crucial variables for assessing the benefits and risks of drugs, but numerical data on utilities may be unreliable or altogether missing. We propose a method to incorporate qualitative information into a probabilistic decision analysis framework for quantitative benefit-risk assessment.

OBJECTIVE

To investigate whether conclusive results can be obtained when the only source of discriminating information on utilities is widely agreed upon qualitative relations, for example, ''sepsis is worse than transient headache'' or ''alleviation of disease is better without than with complications.''

METHOD

We used the structure and probabilities of 3 published models that were originally evaluated based on the standard metric of quality-adjusted life years (QALYs): terfenadine versus chlorpheniramine for the treatment of allergic rhinitis, MCV4 vaccination against meningococcal disease, and alosetron for irritable bowel syndrome. For each model, we identified clinically straightforward qualitative relations among the outcomes. Using Monte Carlo simulations, the resulting utility distributions were then combined with the previously specified probabilities, and the rate of preference in terms of expected utility was determined for each alternative.

RESULTS

Our approach conclusively favored MCV4 vaccination, and it was concordant with the QALY assessments for the MCV4 and terfenadine versus chlorpheniramine case studies. For alosetron, we found a possible unfavorable benefit-risk balance for highly risk-averse patients not identified in the original analysis.

CONCLUSION

Incorporation of widely agreed upon qualitative information into quantitative benefit-risk assessment can provide for conclusive results. The methods presented should prove useful in both population and individual-level assessments, especially when numerical utility data are missing or unreliable, and constraints on time or money preclude its collection.

A formal risk-benefit framework for genomic tests: facilitating the appropriate translation of genomics into clinical practice

PURPOSE

Evaluation of genomic tests is often challenging because of the lack of direct evidence of clinical benefit compared with usual care and unclear evidence requirements. To address these issues, this study presents a risk-benefit framework for assessing the health-related utility of genomic tests.

METHODS

We incorporated approaches from a variety of established fields including decision science, outcomes research, and health technology assessment to develop the framework. Additionally, we considered genomic test stakeholder perspectives and case studies.

RESULTS

We developed a three-tiered framework: first, we use decision-analytic modeling techniques to synthesize data, project incidence of clinical events, and assess uncertainty. Second, we defined the health-related utility of genomic tests as improvement in health outcomes as measured by clinical event rates, life expectancy, and quality-adjusted life-years. Finally, we displayed results using a risk-benefit policy matrix to facilitate the interpretation and implementation of findings from these analyses.

CONCLUSION

A formal risk-benefit framework may accelerate the utilization and practice-based evidence development of genomic tests that pose low risk and offer plausible clinical benefit, while discouraging premature use of tests that provide little benefit or pose significant health risks compared with usual care.

The use of a clinical utility index to compare insomnia compounds: a quantitative basis for benefit-risk assessment

The use of a clinical utility index (CUI) was proposed in order to compare two calcium channel alpha2delta ligands that were in development for the treatment of insomnia. The important attributes included in the CUI were two measures of residual sedation and five measures of efficacy (wake after sleep onset, sleep quality, sleep latency, and sleep stages (stage 1 and stages 3-4)). Dose-response analyses were conducted on each end point, and a sensitivity analysis was conducted to determine a clinically meaningful difference in CUI. Nonparametric bootstrap parameters were used to build confidence intervals (CIs). Peak CUI (80% CI) was 0.345 (0.25-0.43), observed at a dose of approximately 30 mg with the lead compound and 0.436 (0.35-0.52) observed at >600-mg dose for the backup. Although CUI was slightly greater for the backup, peak CUI values were observed at doses that were not considered viable, and therefore development of the ligand was discontinued. The use of the CUI allowed an efficient, quantitative, and transparent decision.