Radiology AI Deployment and Assessment Rubric (RADAR) to bring value-based AI into radiological practice

OBJECTIVE

To provide a comprehensive framework for value assessment of artificial intelligence (AI) in radiology.

METHODS

This paper presents the RADAR framework, which has been adapted from Fryback and Thornbury's imaging efficacy framework to facilitate the valuation of radiology AI from conception to local implementation. Local efficacy has been newly introduced to underscore the importance of appraising an AI technology within its local environment. Furthermore, the RADAR framework is illustrated through a myriad of study designs that help assess value.

RESULTS

RADAR presents a seven-level hierarchy, providing radiologists, researchers, and policymakers with a structured approach to the comprehensive assessment of value in radiology AI. RADAR is designed to be dynamic and meet the different valuation needs throughout the AI's lifecycle. Initial phases like technical and diagnostic efficacy (RADAR-1 and RADAR-2) are assessed pre-clinical deployment via in silico clinical trials and cross-sectional studies. Subsequent stages, spanning from diagnostic thinking to patient outcome efficacy (RADAR-3 to RADAR-5), require clinical integration and are explored via randomized controlled trials and cohort studies. Cost-effectiveness efficacy (RADAR-6) takes a societal perspective on financial feasibility, addressed via health-economic evaluations. The final level, RADAR-7, determines how prior valuations translate locally, evaluated through budget impact analysis, multi-criteria decision analyses, and prospective monitoring.

CONCLUSION

The RADAR framework offers a comprehensive framework for valuing radiology AI. Its layered, hierarchical structure, combined with a focus on local relevance, aligns RADAR seamlessly with the principles of value-based radiology.

CRITICAL RELEVANCE STATEMENT

The RADAR framework advances artificial intelligence in radiology by delineating a much-needed framework for comprehensive valuation.

KEYPOINTS

• Radiology artificial intelligence lacks a comprehensive approach to value assessment. • The RADAR framework provides a dynamic, hierarchical method for thorough valuation of radiology AI. • RADAR advances clinical radiology by bridging the artificial intelligence implementation gap.

Value-based radiology: what is the ESR doing, and what should we do in the future?

Value-based radiology (VBR) is rapidly gaining ground as a means of considering the input of radiology practice into individual and societal healthcare, and represents a welcome move away from older metrics focused on counting studies performed, without consideration of whether these studies contributed positively to patient management or to society as a whole. Intrinsic to the process of considering whether radiology activity confers value is recognising the breadth of involvement of radiology in healthcare delivery; previous ESR and multi-society publications have explored this, and have sought to highlight the many ways in which our specialty contributes to patient welfare. This paper is intended to highlight some current ESR activities which already contribute substantially to value creation and delivery, and to outline a selection of practical steps which could be taken by the ESR in the future to enhance value.

Patient summary

Value-based radiology (VBR) is a conceptual means of looking at the benefits conferred on patients and on society as a whole by provision of radiology services, as opposed to older means of counting numbers of radiology studies performed, without consideration of whether or not those studies contributed overall value. VBR will become increasingly important in the future as a means of determining resources. The ESR has been a leader in advancing VBR concepts and educating radiologists about this novel way of looking at what we do. This paper is designed to highlight current ESR activities which contribute value to healthcare, and to consider other ways in which the ESR could potentially support value enhancement in the future.

ESR concept paper on value-based radiology

The European Society of Radiology (ESR) established a Working Group on Value-Based Imaging (VBI WG) in August 2016 in response to developments in European healthcare systems in general, and the trend within radiology to move from volume- to value-based practice in particular. The value-based healthcare (VBH) concept defines "value" as health outcomes achieved for patients relative to the costs of achieving them. Within this framework, value measurements start at the beginning of therapy; the whole diagnostic process is disregarded, and is considered only if it is the cause of errors or complications. Making the case for a new, multidisciplinary organisation of healthcare delivery centred on the patient, this paper establishes the diagnosis of disease as a first outcome in the interrelated activities of the healthcare chain. Metrics are proposed for measuring the quality of radiologists' diagnoses and the various ways in which radiologists provide value to patients, other medical specialists and healthcare systems at large. The ESR strongly believes value-based radiology (VBR) is a necessary complement to existing VBH concepts. The Society is determined to establish a holistic VBR programme to help European radiologists deal with changes in the evolution from volume- to value-based evaluation of radiological activities. Main Messages • Value-based healthcare defines value as patient's outcome over costs. • The VBH framework disregards the diagnosis as an outcome. • VBH considers diagnosis only if wrong or a cause of complications. • A correct diagnosis is the first outcome that matters to patients. • Metrics to measure radiologists' impacts on patient outcomes are key. • The value provided by radiology is multifaceted, going beyond exam volumes.