Comparison of an oncology clinical decision-support system's recommendations with actual treatment decisions

Objective

IBM^(R) Watson for Oncology (WfO) is a clinical decision-support system (CDSS) that provides evidence-informed therapeutic options to cancer-treating clinicians. A panel of experienced oncologists compared CDSS treatment options to treatment decisions made by clinicians to characterize the quality of CDSS therapeutic options and decisions made in practice.

Methods

This study included patients treated between 1/2017 and 7/2018 for breast, colon, lung, and rectal cancers at Bumrungrad International Hospital (BIH), Thailand. Treatments selected by clinicians were paired with therapeutic options presented by the CDSS and coded to mask the origin of options presented. The panel rated the acceptability of each treatment in the pair by consensus, with acceptability defined as compliant with BIH’s institutional practices. Descriptive statistics characterized the study population and treatment-decision evaluations by cancer type and stage.

Results

Nearly 60% (187) of 313 treatment pairs for breast, lung, colon, and rectal cancers were identical or equally acceptable, with 70% (219) of WfO therapeutic options identical to, or acceptable alternatives to, BIH therapy. In 30% of cases (94), 1 or both treatment options were rated as unacceptable. Of 32 cases where both WfO and BIH options were acceptable, WfO was preferred in 18 cases and BIH in 14 cases. Colorectal cancers exhibited the highest proportion of identical or equally acceptable treatments; stage IV cancers demonstrated the lowest.

Conclusion

This study demonstrates that a system designed in the US to support, rather than replace, cancer-treating clinicians provides therapeutic options which are generally consistent with recommendations from oncologists outside the US.

Physician Perception and Satisfaction with Artificial Intelligence in Cancer Treatment: The Watson for Oncology Experience and Implications for Low-Middle Income Countries (Preprint)

As technology continues to improve, health care systems have the opportunity to use a variety of innovative tools for decision-making, including artificial intelligence (AI) applications. However, there has been little research on the feasibility and efficacy of integrating AI systems into real-world clinical practice, especially from the perspectives of clinicians who use such tools. In this paper, we review physicians’ perceptions of and satisfaction with an AI tool, Watson for Oncology, which is used for the treatment of cancer. Watson for Oncology has been implemented in several different settings, including Brazil, China, India, South Korea, and Mexico. By focusing on the implementation of an AI-based clinical decision support system for oncology, we aim to demonstrate how AI can be both beneficial and challenging for cancer management globally and particularly for low-middle–income countries. By doing so, we hope to highlight the need for additional research on user experience and the unique social, cultural, and political barriers to the successful implementation of AI in low-middle–income countries for cancer care.