Value of information methods for assessing a new diagnostic test

A simple framework to identify optimal cost-effective risk thresholds for a single screen: Comparison to Decision Curve Analysis

Decision Curve Analysis (DCA) is a popular approach for assessing biomarkers and risk models, but does not require costs and thus cannot identify optimal risk thresholds for actions. Full decision analyses can identify optimal thresholds, but typically used methods are complex and often difficult to understand. We develop a simple framework to calculate the Incremental Net Benefit for a single-time screen as a function of costs (for tests and treatments) and effectiveness (life-years gained). We provide simple expressions for the optimal cost-effective risk-threshold and, equally importantly, for the monetary value of life-years gained associated with the risk-threshold. We consider the controversy over the risk-threshold to screen women for mutations in BRCA1/2. Importantly, most, and sometimes even all, of the thresholds identified by DCA are infeasible based on their associated dollars per life-year gained. Our simple framework facilitates sensitivity analyses to cost and effectiveness parameters. The proposed approach estimates optimal risk thresholds in a simple and transparent manner, provides intuition about which quantities are critical, and may serve as a bridge between DCA and a full decision analysis.

Economic evaluation of medical tests at the early phases of development: a systematic review of empirical studies

INTRODUCTION

There is little specific guidance on the implementation of cost-effectiveness modelling at the early stage of test development. The aim of this study was to review the literature in this field to examine the methodologies and tools that have been employed to date.

AREAS COVERED

A systematic review to identify relevant studies in established literature databases. Five studies were identified and included for narrative synthesis. These studies revealed that there is no consistent approach in this growing field. The perspective of patients and the potential for value of information (VOI) to provide information on the value of future research is often overlooked. Test accuracy is an essential consideration, with most studies having described and included all possible test results in their analysis, and conducted extensive sensitivity analyses on important parameters. Headroom analysis was considered in some instances but at the early development stage (not the concept stage).

EXPERT COMMENTARY

The techniques available to modellers that can demonstrate the value of conducting further research and product development (i.e. VOI analysis, headroom analysis) should be better utilized. There is the need for concerted efforts to develop rigorous methodology in this growing field to maximize the value and quality of such analysis.

How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through

An important question in the evaluation of an additional risk prediction marker is how to interpret a small increase in the area under the receiver operating characteristic curve (AUC). Many researchers believe that a change in AUC is a poor metric because it increases only slightly with the addition of a marker with a large odds ratio. Because it is not possible on purely statistical grounds to choose between the odds ratio and AUC, we invoke decision analysis, which incorporates costs and benefits. For example, a timely estimate of the risk of later non-elective operative delivery can help a woman in labor decide if she wants an early elective cesarean section to avoid greater complications from possible later non-elective operative delivery. A basic risk prediction model for later non-elective operative delivery involves only antepartum markers. Because adding intrapartum markers to this risk prediction model increases AUC by 0.02, we questioned whether this small improvement is worthwhile. A key decision-analytic quantity is the risk threshold, here the risk of later non-elective operative delivery at which a patient would be indifferent between an early elective cesarean section and usual care. For a range of risk thresholds, we found that an increase in the net benefit of risk prediction requires collecting intrapartum marker data on 68 to 124 women for every correct prediction of later non-elective operative delivery. Because data collection is non-invasive, this test tradeoff of 68 to 124 is clinically acceptable, indicating the value of adding intrapartum markers to the risk prediction model.