A pan-Canadian survey of peer review practices in radiation oncology

The Future of Safety and Quality in Radiation Oncology

The increasing complexity of radiation therapy treatment presents new potentials for error and suboptimal care. High-performing programs thus not only require adherence to, but also ongoing improvement of, key safety and quality practices. In this article, we review these practices including standardization, risk analysis, peer review, and maintenance of strong safety culture, while also describing recent innovations and promising future directions. We specifically highlight the growing role of artificial intelligence in radiation oncology, both as a tool to deliver safe, high-quality care and as a potential new source of safety challenges.

Radiation Oncology Peer Review in a Community Setting: The Value of Prospective Review

Peer review is an important component of any radiation oncology continuous quality improvement program. While limited guidelines exist, there is no consensus about how peer review should be performed, and large variations exist among different institutions. The purpose of this report is to describe our experience with peer review at a busy Radiation Oncology clinic and to evaluate the difference between prospective and retrospective peer review. We also performed a failure modes and effects analysis (FMEA) of the peer review process. Starting in 2015, every peer review session was tracked, including recommended changes to treatment plans. We reviewed the frequency, types and severity of these changes. A team of physicians and physicists conducted an FMEA of the peer review process. Between April 2015 and June 2020, a total of 3,691 patients were peer-reviewed. Out of those, 1,903 were prospective reviews (51.6%). Plans reviewed before treatment were almost 4.5 times more likely to be changed by peer review than those reviewed after the start of treatment (0.9% vs 0.2%). Plan changes after the start of treatment had a higher severity than changes prior to the start of treatment. FMEA identified several critical components of peer review. While there is no national standard for peer review, it is evident that prospective peer review is preferable. There may be a subconscious reluctance to change plans already underway, which could be a barrier to improving plans with the peer review process. Rather than reviewing in a group setting, it would be ideal to individually assign review tasks that are embedded in the clinical flow, assuring prospective review for all patients prior to final physician approval. Individual review rather than group review may be more candid, due to interpersonal concerns about publicly disagreeing with colleagues.

Peer review quality assurance in stereotactic body radiotherapy planning: the impact of case volume

Purpose:

Peer review is an essential quality assurance component of radiation therapy planning. A growing body of literature has demonstrated substantial rates of suggested plan changes resulting from peer review. There remains a paucity of data on the impact of peer review rounds for stereotactic body radiation therapy (SBRT). We therefore aim to evaluate the outcomes of peer review in this specific patient cohort.

Methods and materials:

We conducted a retrospective review of all SBRT cases that underwent peer review from July 2015 to June 2018 at a single institution. Weekly peer review rounds are grouped according to cancer subsite and attended by radiation oncologists, medical physicists and medical radiation technologists. We prospectively compiled ‘learning moments’, defined as cases with suggested changes or where an educational discussion occurred beyond routine management, and critical errors, defined as errors which could alter clinical outcomes, recorded prospectively during peer review. Plan changes implemented after peer review were documented.

Results:

Nine hundred thirty-four SBRT cases were included. The most common treatment sites were lung (518, 55%), liver (196, 21%) and spine (119, 13%). Learning moments were identified in 161 cases (17%) and translated into plan changes in 28 cases (3%). Two critical errors (0.2%) were identified: an inadequate planning target volume margin and an incorrect image set used for contouring. There was a statistically significantly higher rate of learning moments for lower-volume SBRT sites (defined as ≤30 cases/year) versus higher-volume SBRT sites (29% vs 16%, respectively; p = 0.001).

Conclusions:

Peer review for SBRT cases revealed a low rate of critical errors, but did result in implemented plan changes in 3% of cases, and either educational discussion or suggestions of plan changes in 17% of cases. All SBRT sites appear to benefit from peer review, though lower-volume sites may require particular attention.

The impact of radiation therapist-led structured peer review meetings on compliance to Radiation Oncology Practice Standards

INTRODUCTION

Regular tumour-specific peer review meetings (TPRMs) were established by our group during 2016. A dedicated Quality Assurance Radiation Therapist (QART) was employed in 2018 to co-ordinate the meetings and for each patient, complete the Peer Review Audit Tool (PRAT) of the Royal Australian and New Zealand College of Radiologists (RANZCR). The aim of the current quality assurance study was to investigate the impact of the TPRMs and appointment of the QART on compliance to relevant RANZCR Radiation Oncology Practice Standards (ROPS).

METHODS

Tumour-specific peer review meetings for eight tumour sites were assessed across our group's three hospitals from January 2017 to December 2019. Data from meetings were collected using the PRAT or from paper-based minutes and assessed against four ROPS (ROPS 3, 4, 8 and 9). Compliance with each of the four standards was measured by presence of the required documentation and presentation at TPRM, as recorded by the PRAT.

RESULTS

There was an increase in the overall number of peer review cases audited from 173 in the 2017 calendar year to 469 in 2018 and 619 in 2019, representing 7%, 18% and 22% of all treatment courses started during these years, respectively. Staging was the most incompletely documented item across all years for audited patients. The request for radiation treatment plan modifications increased year-on-year: modifications were requested for 5% of plans in 2017 (8/172), 18% in 2018 (81/452) and 19% (119/619) in 2019.

CONCLUSION

This study has shown that an increase in the number of cases for peer-review audit corresponded to the QART-facilitated TPRMs. Application of the PRAT has identified radiation treatment plan modifications that would otherwise go undetected and without opportunity to improve the quality of patients' treatment or avoid harm.

Challenges in the target volume definition of lung cancer radiotherapy

Radiotherapy, with or without systemic treatment has an important role in the management of lung cancer. In order to deliver the treatment accurately, the clinician must precisely outline the gross tumour volume (GTV), mostly on computed tomography (CT) images. However, due to the limited contrast between tumour and non-malignant changes in the lung tissue, it can be difficult to distinguish the tumour boundaries on CT images leading to large interobserver variation and differences in interpretation. Therefore the definition of the GTV has often been described as the weakest link in radiotherapy with its inaccuracy potentially leading to missing the tumour or unnecessarily irradiating normal tissue. In this article, we review the various techniques that can be used to reduce delineation uncertainties in lung cancer.

A Comprehensive Analysis of a Prospective Multidisciplinary Peer Review Process Before Radiation Therapy Simulation

PURPOSE

Although peer review in radiation oncology (RO) has been recommended to improve quality of care, an analysis of modifications resulting from an RO multidisciplinary presimulation standardized review process has yet to be empirically demonstrated.

METHODS AND MATERIALS

A standardized simulation directive was used for patients undergoing simulation for external beam radiation therapy at a single tertiary care institution. The simulation directives were presented, and all aspects were reviewed by representatives from key RO disciplines. Modifications to the original directives were prospectively captured in a quality improvement registry. Association between key variables and the incidence of modifications were performed using Fisher exact test and t test.

RESULTS

A registry of 500 consecutive simulations for patients undergoing radiation therapy was reviewed. A median of 105 simulations occurred per month. All simulation directives were entered by a physician a median of 3 days before simulation (range, 1-76 days). The treatment intent was curative for 269 patients (53.8%), palliative for 203 patients (40.6%), and benign for 3 patients (0.6%). Twenty-five (5%) patients did not have a treatment intent selected. Based on RO multidisciplinary review, 105 directives (21%) were modified from the original intent, with 29 (5.8%) requiring more than 1 modification. A total of 149 modifications were made and categorized as changes to patient positioning and immobilization (n = 100, 20%), treatment site and care path (n = 34, 6.8%), simulation coordination activities (n = 6, 1.2%), and treatment technique and planning instructions (n = 9, 1.8%). A higher proportion of modifications occurred at the time of multidisciplinary review in patients receiving more complex treatments (intensity modulated radiation therapy/stereotactic radiosurgery/stereotactic body radiation therapy [IMRT/SRS/SBRT] vs 3-dimensional radiation therapy [3DCRT] radiation therapy, 25% vs 16%, P < .025).

CONCLUSIONS

Given the complexity of radiation therapy simulation, standardization of directives with prospective RO multidisciplinary presimulation peer review is critical to optimizing department processes and reducing errors. Approximately 1 in 5 patients benefits from this peer review process, especially patients treated with IMRT/SRS/SBRT.

A Blinded, Prospective Study of Error Detection During Physician Chart Rounds in Radiation Oncology

PURPOSE

Peer review during physician chart rounds is a major quality assurance and patient safety step in radiation oncology. However, the effectiveness of chart rounds in detecting problematic treatment plans is unknown. We performed a prospective blinded study of error detection at chart rounds to clarify the effectiveness of this quality assurance step.

METHODS AND MATERIALS

Radiation Oncology Incident Learning System publications were queried for problematic plans approved for treatment that would be detectable at chart rounds. A resident physician, physicist, and dosimetrist collaboratively generated 20 treatment plans with simulated errors identical in nature to those reported to the Radiation Oncology Incident Learning System. These were inserted randomly into weekly chart rounds over 9 weeks, with a median of 2 problematic plans presented per chart rounds (range, 1-4). Data were collected on detection, attendance, length, and number of cases presented at chart rounds. Data were analyzed using descriptive statistics and univariable logistic regression with odds ratios.

RESULTS

The median length of chart rounds over the study period was 60 minutes (range, 42-79); median number of cases presented per chart rounds was 45 (range, 38-50). The overall detection rate was 55% (11 of 20). Detection rates were higher for cases presented earlier in chart rounds: 75% versus 25% of problematic plans were detected within 30 minutes of start of chart rounds versus after 30 minutes (odds ratio, 0.11; 95% confidence interval, 0.01-0.88; P = .037). Detection rates showed a trend toward increase during the study period but this was not significant: 33% in weeks 1 to 5 and 73% during weeks 6 to 9 (5.3; 95% confidence interval, 0.78-36; P = .08).

CONCLUSIONS

The detection of clinically significant problematic plans during chart rounds could be significantly improved. Problematic plans are more frequently detected earlier in chart rounds and inserting such plans into chart rounds may enhance detection; however, larger studies are needed to confirm these findings. A multi-institutional study is planned.

100% peer review in radiation oncology: is it feasible?

Purpose

Peer review has been proposed as a strategy to ensure patient safety and plan quality in radiation oncology. Despite its potential benefits, barriers commonly exist to its optimal implementation in daily clinical routine. Our purpose is to analyze peer-review process at our institution.

Methods and materials

Based on our group peer-review process, we quantified the rate of plan changes, time and resources needed for this process. Prospectively, data on cases presented at our institutional peer-review conference attended by physicians, resident physicians and physicists were collected. Items such as time to present per case, type of patient (adult or pediatric), treatment intent, dose, aimed technique, disease location and receipt of previous radiation were gathered. Cases were then analyzed to determine the rate of major change, minor change and plan rejection after presentation as well as the median time per session.

Results

Over a period of 4 weeks, 148 cases were reviewed. Median of attendants was six physicians, three in-training-physicians and one physicist. Median time per session was 38 (4–72) minutes. 59.5% of cases presented in 1–4 min, 32.4% in 5–9 min and 8.1% in ≥ 10 min. 79.1% of cases were accepted without changes, 11.5% with minor changes, 6% with major changes and 3.4% were rejected with indication of new presentation. Most frequent reason of change was contouring corrections (53.8%) followed by dose or fractionation (26.9%).

Conclusion

Everyday group consensus peer review is an efficient manner to recollect clinical and technical data of cases presented to ensure quality radiation care before initiation of treatment as well as ensuring department quality in a feedback team environment. This model is feasible within the normal operation of every radiation oncology Department.

Safety First: Developing and Deploying a System to Promote Safety and Quality in Your Clinic

The terms "safety and quality" (SAQ) have become inextricably linked, highly used terms that together encompass a wide range of parameters within medical departments. Safety has always been a priority in radiation oncology; quality assurance has been foundational to our practice. Despite this increased focus and attention on SAQ, the "what" of SAQ remains ill-defined, largely because of the vast number of indicators that fall under this umbrella. Similarly, the "how" of developing and maintaining the highest standards of SAQ is not formulaic and varies based on the unique setting of individual practices. There are several excellent resources available to inform SAQ in radiation oncology, including the American Society for Radiation Oncology's "Safety Is No Accident," which provides an overview of safety and quality standards and resources. This review is intended as a brief summary of key considerations, with the goal of providing a practical framework and context for improving or developing a SAQ program in radiation oncology practices. We believe that the following 10 key elements, drawn from numerous reports that have appeared over the last decade examining this topic, should be considered when conceptualizing a practice-based approach to SAQ: establishing a strong safety culture; establishing a structured program for safety and quality; establishing up-to-date, relevant, and accessible policies and procedures; a system for peer review; systems to assess and reduce risk; an educational program focused on safety and quality; development and review of meaningful quality metrics; utilization of a physics quality control system; well-defined models for staffing, training, and professional development; and finally, validation from external bodies via accreditations and audits. These 10 items are addressed herein.

Prospective Peer Review in Radiation Therapy Treatment Planning: Long-Term Results From a Longitudinal Study

PURPOSE

To present the longitudinal results of a prospective peer review evaluation system (PES) before treatment planning.

METHODS AND MATERIALS

All cases undergoing radiation therapy (RT) at high-volume academic institutions were graded in daily prospective multidisciplinary contouring rounds (CRs). The clinical suitability for RT, prescription, contours, and written directives were peer reviewed, compared with departmental care pathways, and recorded in a prospective database. Grades were assigned as follows: A (score 4.0) = no deficiencies; B (3.0) = minor modifications of the planning target volume, organs at risk, written directives, or a prescription/care pathway mismatch; and C (2.0) = incomplete target volume or organ-at-risk contours, unsuitable use or inappropriate planned administration of RT, significant contour modifications, prescription changes, or laterality modifications. Information was pooled to determine pretreatment planning work performance by assigning a grade point average (GPA) for each physician as well as compositely.

RESULTS

A total of 11,843 treatment plans from 7854 patients were reviewed using the PES from September 2013 to May 2018. Twenty-seven point nine percent of cases (n = 3303) required modifications before treatment planning commenced. The overall breakdown of grades was 72.1% As, 21.7% Bs, and 6.2% Cs. The median physician CR GPA was 3.60 (average 3.7) with a range of 3.0 to 3.9. Seventy-five percent of physicians demonstrated improvement of their CR GPA since inception of the program, and all physicians demonstrated a drop in the percentage of cases that were assigned a grade of C.

CONCLUSIONS

The PES can transparently quantify clinical performance in a single metric. The PES was impactful, with 75% of physicians demonstrating improvement in their CR GPA over time. In contrast to traditional chart rounds, this peer review was meaningful when done before planning commenced, a trend that was observed throughout the study period. Twenty-seven point nine percent of all cases required modification before starting treatment planning, and 6.2% of cases required significant remediation.

Outcomes of Peer Review for Radiotherapy Treatment Plans With Palliative Intent

PURPOSE

Peer review of a proposed treatment plan is increasingly recognized as an important quality activity in radiation medicine. Although peer review has been emphasized in the curative setting, applying peer review for treatment plans that have palliative intent is receiving increased attention. This study reports peer review outcomes for a regional cancer center that applied routine interprofessional peer review as a standard practice for palliative radiotherapy.

METHODS AND MATERIALS

Peer review outcomes for palliative radiotherapy plans were recorded prospectively for patients who began radiotherapy between October 1, 2015, and September 30, 2017. Recommended and implemented changes were recorded. The content of detailed discussions was recorded to gain insight into the complexities of palliative treatment plans considered during peer review.

RESULTS

Peer review outcomes were reviewed for 1,413 treatment plans with palliative intent. The proportions of detailed discussions and changes recommended were found to be 139 (9.8%) and 29 (2.1%), respectively. The content of detailed discussions and changes recommended was categorized. Major changes represented 75.9% of recommended changes, of which 84.2% were implemented clinically.

CONCLUSION

Many complexities exist that are specific to palliative radiotherapy. Interprofessional peer review provides a forum for these complexities to be openly discussed and is an important activity to optimize the quality of care for patients with treatment plans that have palliative intent.

Analysis of a real time group consensus peer review process in radiation oncology: an evaluation of effectiveness and feasibility

Background

Peer review systems within radiation oncology are important to ensure quality radiation care. Several individualized methods for radiation oncology peer review have been described. However, despite the importance of peer review in radiation oncology barriers may exist to its effective implementation in practice. The purpose of this study was to quantify the rate of plan changes based on our group peer review process as well as the quantify amount of time and resources needed for this process.

Methods

Data on cases presented in our institutional group consensus peer review conference were prospectively collected. Cases were then retrospectively analyzed to determine the rate of major change (plan rejection) and any change in plans after presentation as well as the median time of presentation. Univariable logistic regression was used to determine factors associated with major change and any change.

Results

There were 73 cases reviewed over a period of 11 weeks. The rate of major change was 8.2% and the rate of any change was 23.3%. The majority of plans (53.4%) were presented in 6–10 min. Overall, the mean time of presentation was 8 min. On univariable logistic regression, volumetric modulated arc therapy plans were less likely to undergo a plan change but otherwise there were no factors significantly associated with major plan change or any type of change.

Conclusion

Group consensus peer review allows for a large amount of informative clinical and technical data to be presented per case prior to the initiation of radiation treatment in a thorough yet efficient manner to ensure plan quality and patient safety.

Impact of Multi-Institutional Prospective Peer Review on Target and Organ-at-Risk Delineation in Radiation Therapy

PURPOSE

Peer review is an essential component of quality assurance programs in radiation oncology. The purpose of this work was to assess whether peer reviewers recommend expansion or reduction of planning target volumes (PTVs) and organs at risk (OARs) in prospective multidisciplinary daily contour rounds.

METHODS AND MATERIALS

The peer group evaluated the appropriateness of PTVs and OARs for each case according to evidence-based departmental directives. We reviewed 7645 cases that presented between September 2013 and March 2017. We isolated recommendations for PTV/OAR modification and classified each as expansion, reduction, both, or indeterminate. Recommendations were analyzed by technique, site, and physician experience.

RESULTS

Eight junior and 7 senior radiation oncologists were included. PTV or OAR modifications were recommended for 750 of 7645 prescriptions (9.7%). The peer group recommended PTV modifications for 534 prescriptions (7.0%): There were 309 expansions (57.9%), 115 reductions (21.5%), 15 both (2.8%), and 95 indeterminate (17.8%). Reasons for PTV expansions included increased nodal coverage and inadequate margins as a result of motion. The peer group recommended OAR modifications for 216 prescriptions (2.8%): There were 102 expansions (47.2%), 23 reductions (10.6%), 2 both (0.9%), and 89 indeterminate (41.2%). Reasons for OAR expansions included missing critical structures and inadequate extent as per departmental standardization. Head and neck represented the largest percentage of PTV recommendations (28.8%). Intensity modulated radiation therapy plans received the most PTV and OAR recommendations (66.8% and 74.5%, respectively). The recommendation rate for senior and junior faculty was 43% and 28%, respectively.

CONCLUSIONS

Peer review resulted in recommendations for PTV or OAR change for approximately 10% of cases. Expansions of PTV were recommended >2.5 times more often than reductions and >3 times more often than OAR expansions. This general trend was identified for treatment technique, site, and physician experience. Prospective peer review could yield systematically larger volumes, which could affect multicenter clinical trials.

Group consensus peer review in radiation oncology: commitment to quality

Background

Peer review, especially prospective peer review, has been supported by professional organizations as an important element in optimal Radiation Oncology practice based on its demonstration of efficacy at detecting and preventing errors prior to patient treatment. Implementation of peer review is not without barriers, but solutions do exist to mitigate or eliminate some of those barriers.

Methods

Peer review practice at our institution involves three key elements: new patient conference, treatment planning conference, and chart rounds. The treatment planning conference is an adaptation of the group consensus peer review model from radiology which utilizes a group of peers reviewing each treatment plan prior to implementation. The peer group in radiation oncology includes Radiation Oncologists, Physician Residents, Medical Physicists, Dosimetrists, and Therapists. Thus, technical and clinical aspects of each plan are evaluated simultaneously.

Results

Though peer review is held in high regard in Radiation Oncology, many barriers commonly exist preventing optimal implementation such as time intensiveness, repetition, and distraction from clinic time with patients. Through the use of automated review tools and commitment by individuals and administration in regards to staffing, scheduling, and responsibilities, these barriers have been mitigated to implement this Group Consensus Peer Review model into a Radiation Oncology Clinic.

Conclusion

A Group Consensus Peer Review model has been implemented with strategies to address common barriers to effective and efficient peer review.

Creation of an educational quality improvement program for radiation oncology residents

PURPOSE

Quality improvement (QI) is a pillar of good clinical governance and is at the center of modern health care. The Royal College of Physicians and Surgeons of Canada mandated, in CanMeds 2015, that QI should be taught and the competencies assessed in all postgraduate residency programs. The objective is to report on the feasibility and impact of teaching QI to radiation oncology residents at a single institution.

METHODS AND MATERIALS

A QI team consisting of a clinical fellow, 3 staff physicians, and an expert in QI methods was created within our Department of Radiation Oncology. QI teaching took place in a longitudinal manner, with approximately 12 hours of direct faculty teaching. A mandatory curriculum divided into foundation and intermediate and advanced competencies was devised. Phase 1 teaching, delivered during 2 academic half-days, consisted of didactic lectures, practical workshops, and self-directed online modules. Phase 2 required intermediate-year residents to complete a 9-month QI project. A QI day hosted by the department invited QI experts to teach and enabled residents to present their work, with merit prizes awarded. Our program evaluation used validated assessment tools (self-assessment, QI knowledge-based assessments, and balanced score cards) before and after curriculum implementation and answers quantified using satisfaction indices (SI).

RESULTS

Subjective and objective assessments demonstrated improvements in residents' QI knowledge acquisition following curriculum implementation. Those who had completed a project (n = 4) had greater confidence with QI methodology compared with those who had completed phase 1 alone (n = 2) (mean SI, 53% precurriculum to 66.5% and 90%). The majority lacked previous QI teaching and knowledge, but learner attitudes improved (SI, 50%-70%) and 91% of colleagues were enthusiastic about the program being implemented.

CONCLUSION

We have demonstrated that implementation of a QI curriculum for radiation oncology residents is feasible and that early results suggesting improvements of attitude and knowledge are positive. We anticipate that the QI skills gained will enable the residents to elevate the quality of their practice throughout their subsequent careers.

Safety Considerations in Stereotactic Body Radiation Therapy

Although many error pathways are common to both stereotactic body radiation therapy (SBRT) and conventional radiation therapy, SBRT presents a special set of challenges including short treatment courses and high-doses, an enhanced reliance on imaging, technical challenges associated with commissioning, special resource requirements for staff and training, and workflow differences. Emerging data also suggest that errors occur at a higher rate in SBRT treatments. Furthermore, when errors do occur they often have a greater effect on SBRT treatments. Given these challenges, it is important to understand and employ systematic approaches to ensure the quality and safety of SBRT treatment. Here, we outline the pathways by which error can occur in SBRT, illustrated through a series of case studies, and highlight 9 specific well-established tools to either reduce error or minimize its effect to the patient or both.