Hillis SL, Berbaum KS, Metz CE. Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis.
Acad Radiol 2008;
15:647-61. [PMID:
18423323 DOI:
10.1016/j.acra.2007.12.015]
[Citation(s) in RCA: 152] [Impact Index Per Article: 9.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2007] [Revised: 12/21/2007] [Accepted: 12/22/2007] [Indexed: 10/22/2022]
Abstract
RATIONALE AND OBJECTIVES
The Dorfman-Berbaum-Metz (DBM) method has been one of the most popular methods for analyzing multireader receiver-operating characteristic (ROC) studies since it was proposed in 1992. Despite its popularity, the original procedure has several drawbacks: it is limited to jackknife accuracy estimates, it is substantially conservative, and it is not based on a satisfactory conceptual or theoretical model. Recently, solutions to these problems have been presented in three papers. Our purpose is to summarize and provide an overview of these recent developments.
MATERIALS AND METHODS
We present and discuss the recently proposed solutions for the various drawbacks of the original DBM method.
RESULTS
We compare the solutions in a simulation study and find that they result in improved performance for the DBM procedure. We also compare the solutions using two real data studies and find that the modified DBM procedure that incorporates these solutions yields more significant results and clearer interpretations of the variance component parameters than the original DBM procedure.
CONCLUSIONS
We recommend using the modified DBM procedure that incorporates the recent developments.
Collapse