Clinician based decision tool to guide recommended interval between colonoscopies: development and evaluation pilot study

Establishment of a comprehensive set of fact sheets for cancer predisposition genes for medical oncologists practicing cancer genome profiling

BACKGROUND

Comprehensive genomic profiling (CGP) is widely performed worldwide, increasing opportunities for medical oncologists to explain cancer predisposition at the time of informed consent and return of results. How medical oncologists communicate about (suspected) cancer predisposition genes is a key factor in referring patients for consultation with genetic services. In this study, we developed a set of fact sheets on cancer predisposition genes to support medical oncologists in their practice under the nationwide cancer genome screening project MONSTAR-SCREEN-2 study in Japan.

METHODS

The Genetic Specialist Committee, comprising clinical geneticists, genetic counselors, bioinformaticians, and medical oncologists, drafted the fact sheet and external Genetic Experts reviewed its elements and contents. A fact sheet evaluation survey was conducted one year after the fact sheet was completed and distributed to medical oncologists at the National Cancer Center Hospital East.

RESULTS

The content of the fact sheet included an overview of diseases, inheritance, family impact, lifetime risk, and surveillance. In the evaluation survey, 83.3% of respondents rated it as "useful." Notably, the sections "What is genetic counseling" (100%) and "Lifetime risk" (94.4%) received high ratings.

CONCLUSION

Our study suggests that a fact sheet developed by the Genetic Specialist Committee may help medical oncologists explain CGP results and connect patients to genetic services. It also functions as an educational resource that requires periodic updates and is in line with revisions to the guidelines.

Colorectal Cancer Prevention: Is Chat Generative Pretrained Transformer (Chat GPT) ready to Assist Physicians in Determining Appropriate Screening and Surveillance Recommendations?

OBJECTIVE

To determine whether a publicly available advanced language model could help determine appropriate colorectal cancer (CRC) screening and surveillance recommendations.

BACKGROUND

Poor physician knowledge or inability to accurately recall recommendations might affect adherence to CRC screening guidelines. Adoption of newer technologies can help improve the delivery of such preventive care services.

METHODS

An assessment with 10 multiple choice questions, including 5 CRC screening and 5 CRC surveillance clinical vignettes, was inputted into chat generative pretrained transformer (ChatGPT) 3.5 in 4 separate sessions. Responses were recorded and screened for accuracy to determine the reliability of this tool. The mean number of correct answers was then compared against a control group of gastroenterologists and colorectal surgeons answering the same questions with and without the help of a previously validated CRC screening mobile app.

RESULTS

The average overall performance of ChatGPT was 45%. The mean number of correct answers was 2.75 (95% CI: 2.26-3.24), 1.75 (95% CI: 1.26-2.24), and 4.5 (95% CI: 3.93-5.07) for screening, surveillance, and total questions, respectively. ChatGPT showed inconsistency and gave a different answer in 4 questions among the different sessions. A total of 238 physicians also responded to the assessment; 123 (51.7%) without and 115 (48.3%) with the mobile app. The mean number of total correct answers of ChatGPT was significantly lower than those of physicians without [5.62 (95% CI: 5.32-5.92)] and with the mobile app [7.71 (95% CI: 7.39-8.03); P < 0.001].

CONCLUSIONS

Large language models developed with artificial intelligence require further refinements to serve as reliable assistants in clinical practice.

Optimizing Timing of Follow-Up Colonoscopy: A Pilot Cluster Randomized Trial of a Knowledge Translation Tool

BACKGROUND

Endoscopists have low adherence to guideline-recommended colonoscopy surveillance intervals. We performed a cluster-randomized single-blind pilot trial in Winnipeg, Canada, to assess the effectiveness of a newly developed digital application tool that computes guideline-recommended follow-up intervals.

METHODS

Participant endoscopists were randomized to either receive access to the digital application (intervention group) or not receive access (control group). Pathology reports and final recommendations for colonoscopies performed in the 1-4 months before randomization and 3-7 months postrandomization were extracted. Generalized estimating equation models were used to determine whether the access to the digital application predicted guideline congruence.

RESULTS

We included 15 endoscopists in the intervention group and 14 in the control group (of 42 eligible endoscopists in the city), with 343 patients undergoing colonoscopy before randomization and 311 postrandomization. Endoscopists who received the application made guideline-congruent recommendations 67.6% of the time before randomization and 76.1% of the time after randomization. Endoscopists in the control group made guideline-congruent recommendations 72.4% and 72.9% of the time before and after randomization, respectively. Endoscopists in the intervention group trended to have an increase in guideline adherence comparing postintervention with preintervention (odds ratio [OR]: 1.50, 95% confidence interval [CI] 0.82-2.74). By contrast, the control group had no change in guideline adherence (OR: 1.07, 95% CI 0.50-2.29). Endoscopists in the intervention group with less than median guideline congruence prerandomization had a significant increase in guideline-congruent recommendations postrandomization.

DISCUSSION

An application that provides colonoscopy surveillance intervals may help endoscopists with guideline congruence, especially those with a lower preintervention congruence with guideline recommendations ( ClincialTrials.gov number, NCT04889352).

Improving Concordance Between Clinicians With Australian Guidelines for Bowel Cancer Prevention Using a Digital Application: Randomized Controlled Crossover Study

BACKGROUND

Australia's bowel cancer prevention guidelines, following a recent revision, are among the most complex in the world. Detailed decision tables outline screening or surveillance recommendations for 230 case scenarios alongside cessation recommendations for older patients. While these guidelines can help better allocate limited colonoscopy resources, their increasing complexity may limit their adoption and potential benefits. Therefore, tools to support clinicians in navigating these guidelines could be essential for national bowel cancer prevention efforts. Digital applications (DAs) represent a potentially inexpensive and scalable solution but are yet to be tested for this purpose.

OBJECTIVE

This study aims to assess whether a DA could increase clinician adherence to Australia's new colorectal cancer screening and surveillance guidelines and determine whether improved usability correlates with greater conformance to guidelines.

METHODS

As part of a randomized controlled crossover study, we created a clinical vignette quiz to evaluate the efficacy of a DA in comparison with the standard resource (SR) for making screening and surveillance decisions. Briefings were provided to study participants, which were tailored to their level of familiarity with the guidelines. We measured the adherence of clinicians according to their number of guideline-concordant responses to the scenarios in the quiz using either the DA or the SR. The maximum score was 18, with higher scores indicating improved adherence. We also tested the DA's usability using the System Usability Scale.

RESULTS

Of 117 participants, 80 were included in the final analysis. Using the SR, the adherence of participants was rated a median (IQR) score of 10 (7.75-13) out of 18. The participants' adherence improved by 40% (relative risk 1.4, P<.001) when using the DA, reaching a median (IQR) score of 14 (12-17) out of 18. The DA was rated highly for usability with a median (IQR) score of 90 (72.5-95) and ranked in the 96th percentile of systems. There was a moderate correlation between the usability of the DA and better adherence (rs=0.4; P<.001). No differences between the adherence of specialists and nonspecialists were found, either with the SR (10 vs 9; P=.47) or with the DA (13 vs 15; P=.24). There was no significant association between participants who were less adherent with the DA (n=17) and their age (P=.06), experience with decision support tools (P=.51), or academic involvement with a university (P=.39).

CONCLUSIONS

DAs can significantly improve the adoption of complex Australian bowel cancer prevention guidelines. As screening and surveillance guidelines become increasingly complex and personalized, these tools will be crucial to help clinicians accurately determine the most appropriate recommendations for their patients. Additional research to understand why some practitioners perform worse with DAs is required. Further improvements in application usability may optimize guideline concordance further.