Radiology workflow for RECIST assessment in clinical trials: Can we reconcile time-efficiency and quality?

Artificial intelligence-based automated matching of pulmonary nodules on follow-up chest CT

BACKGROUND

The growing demand for follow-up imaging highlights the need for tools supporting the assessment of pulmonary nodules over time. We evaluated the performance of an artificial intelligence (AI)-based system for automated nodule matching.

METHODS

In this single-center study, patients with nodules and ≤ 2 chest computed tomography (CT) examinations were retrospectively selected. An AI-based algorithm was used for automated nodule detection and matching. The matching rate and the causes for incorrect matching were evaluated for the ten largest lesions (5-30 mm in diameter) registered on baseline CT. The dependence of the matching rate on nodule number and localization was also analyzed.

RESULTS

One hundred patients (46 females), with a median age of 62 years (interquartile range 57-69), and 253 CTs were included. Focusing on the ten largest lesions, 1,141 lesions were identified, of which 36 (3.2%) were other structures incorrectly identified as nodules (false-positives). Of the 1,105 identified nodules, 964 (87.2%) were correctly detected and matched. The matching rate for nodules registered in both baseline and follow-up scans was 97.8%. The matching rate per case ranged 80.0-100.0% (median 90.0%). Correct matching rate decreased in follow-up examinations to over 50 nodules (p = 0.003), with an overrepresentation of missed matching. Matching rates were higher in parenchymal (91.8%), peripheral (84.4%), and juxtavascular (82.4%) nodules than in juxtaphrenic nodules (71.1%) (p < 0.001). Missed matching was overrepresented in juxtavascular, and incorrect assignment in juxtaphrenic nodules.

CONCLUSION

The correct automated-matching rate of metastatic pulmonary nodules in follow-up examinations was high, but it depends on localization and a number of nodules.

RELEVANCE STATEMENT

The algorithm enables precise follow-up matching of pulmonary nodules, potentially providing a solid basis for standardized and accurate evaluations. Understanding the algorithm's strengths and weaknesses based on nodule localization and number enhances the interpretation of AI-based results.

KEY POINTS

The AI algorithm achieved a correct nodule matching rate of 87.2% and up to 97.8% when considering nodules detected in both baseline and follow-up scans. Matching accuracy depended on nodule number and localization. This algorithm has the potential to support response evaluation criteria in solid tumor-based evaluations in clinical practice.

Tumour measurements on imaging for clinical trial: A national picture of service provision

BACKGROUND

Radiological response evaluation metrics such as RECIST 1.1 inform critical endpoints in oncology trials. The UK was the 6th highest recruiter into oncology trials worldwide between 1999 and 2022, with almost 9000 oncology trials registered during the same period. However, the provision of tumour measurements for oncology trials is often ad hoc and patchy across the NHS. The aim of this work was to understand the barriers to providing an effective imaging tumour measurement service, gain insight into service delivery models and consider the successes and challenges from the perspective of both service providers and end users.

METHODS

An electronic survey was distributed to those who provide tumour measurement response review for clinical trials (service providers) and those that request and use such measurements in trial activities (service users).

RESULTS

Responses from 35 sites demonstrated substantial variation in service provision across the UK. Despite workforce pressures, service is largely delivered through radiologists with a minority utilising radiographer role extension. Only 20% of the service providers had dedicated training and 29% received robust financial reimbursement.

DISCUSSION

Service variation is likely a consequence of limited training, education and infrastructure to support robust service, compounded by increasing radiology workload and workforce pressures.

Dedicated software to harmonize the follow-up of oncological patients

Objective

To test and evaluate a sofware dedicated to the follow-up of oncological CT scans for potential use in the Radiology department.

Materials and methods

In this retrospective study, 37 oncological patients with baseline and follow-up CT scans were reinterpreted using a dedicated software. Baseline CT scans were chosen from the imaging reports available in our PACS (picture archiving and communicatin systems). Follow-up interpretations were independently assessed with the software. We evaluated the target lesion sums and the tumor response based on RECIST 1.1 (Response Evaluation Criteria in Solid Tumors).

Results

There was no significant difference in the target lesion sums and the tumor response assessments between the PACS data and the imaging software. There was no over or underestimation of the disease with the software. There was a sigificant deviation (progression versus stability) in three cases. For two patients, this difference was related to the evaluation of the response of non-target lesions. The difference in the third patient was due to comparison with a previous CT scan than to the baseline exam. There was a miscalculation in 13 % of the reports and in 28 % of the cases the examination was compared to the previous CT scan. Finally, the tumor response was not detailed in 43 % of the follow-up reports.

Conclusion

The use of dedicated oncology monitoring software could help in reducing intepretation time and in limiting human errors.

Assessing the performance of AI-assisted technicians in liver segmentation, Couinaud division, and lesion detection: a pilot study

BACKGROUND

In patients with primary and secondary liver cancer, the number and sizes of lesions, their locations within the Couinaud segments, and the volume and health status of the future liver remnant are key for informing treatment planning. Currently this is performed manually, generally by trained radiologists, who are seeing an inexorable growth in their workload. Integrating artificial intelligence (AI) and non-radiologist personnel into the workflow potentially addresses the increasing workload without sacrificing accuracy. This study evaluated the accuracy of non-radiologist technicians in liver cancer imaging compared with radiologists, both assisted by AI.

METHODS

Non-contrast T1-weighted MRI data from 18 colorectal liver metastasis patients were analyzed using an AI-enabled decision support tool that enables non-radiology trained technicians to perform key liver measurements. Three non-radiologist, experienced operators and three radiologists performed whole liver segmentation, Couinaud segment segmentation, and the detection and measurements of lesions aided by AI-generated delineations. Agreement between radiologists and non-radiologists was assessed using the intraclass correlation coefficient (ICC). Two additional radiologists adjudicated any lesion detection discrepancies.

RESULTS

Whole liver volume showed high levels of agreement between the non-radiologist and radiologist groups (ICC = 0.99). The Couinaud segment volumetry ICC range was 0.77-0.96. Both groups identified the same 41 lesions. As well, the non-radiologist group identified seven more structures which were also confirmed as lesions by the adjudicators. Lesion diameter categorization agreement was 90%, Couinaud localization 91.9%. Within-group variability was comparable for lesion measurements.

CONCLUSION

With AI assistance, non-radiologist experienced operators showed good agreement with radiologists for quantifying whole liver volume, Couinaud segment volume, and the detection and measurement of lesions in patients with known liver cancer. This AI-assisted non-radiologist approach has potential to reduce the stress on radiologists without compromising accuracy.

Reliability of Automated RECIST 1.1 and Volumetric RECIST Target Lesion Response Evaluation in Follow-Up CT-A Multi-Center, Multi-Observer Reading Study

OBJECTIVES

To evaluate the performance of a custom-made convolutional neural network (CNN) algorithm for fully automated lesion tracking and segmentation, as well as RECIST 1.1 evaluation, in longitudinal computed tomography (CT) studies compared to a manual Response Evaluation Criteria in Solid Tumors (RECIST 1.1) evaluation performed by three radiologists.

METHODS

Baseline and follow-up CTs of patients with stage IV melanoma (n = 58) was investigated in a retrospective reading study. Three radiologists performed manual measurements of metastatic lesions. Fully automated segmentations were generated, and diameters and volumes were computed from the segmentation results, with subsequent RECIST 1.1 evaluation. We measured (1) the intra- and inter-reader variability in the manual diameter measurements, (2) the agreement between manual and automated diameter measurements, as well as the resulting RECIST 1.1 categories, and (3) the agreement between the RECIST 1.1 categories derived from automated diameter measurement compared to automated volume measurements.

RESULTS

In total, 114 target lesions were measured at baseline and follow-up. The intraclass correlation coefficients (ICCs) for the intra- and inter-reader reliability of the diameter measurements were excellent, being >0.90 for all readers. There was moderate to almost perfect agreement when comparing the timepoint response category derived from the mean manual diameter measurements from all three readers with those derived from automated diameter measurements (Cohen's k 0.67-0.76). The agreement between the manual and automated volumetric timepoint responses was substantial (Fleiss' k 0.66-0.68) and that between the automated diameter and volume timepoint responses was substantial to almost perfect (Cohen's k 0.81).

CONCLUSIONS

The automated diameter measurement of preselected target lesions in follow-up CT is reliable and can potentially help to accelerate RECIST evaluation.

The ins and outs of errors in oncology imaging: the DAC framework for radiologists

With the increasingly central role of imaging in medical diagnosis, understanding and monitoring radiological errors has become essential. In the field of oncology, the severity of the disease makes radiological error more visible, with both individual consequences and public health issues. The quantitative trend radiology allows to consider the diagnostic task as a problem of classification supported by the latest neurocognitive theories in explaining decision making errors, this purposeful model provides an actionable framework to support root cause analysis of diagnostic errors in radiology and envision corresponding risk-management strategies. The D for Data, A for Analysis and C for Communication are the three drivers of errors and we propose a practical toolbox for our colleagues to prevent individual and systemic sources of error.

Improvement of assessment in surrogate endpoint and safety outcome of single-arm trials for anticancer drugs

INTRODUCTION

Single-arm trials (SATs) and surrogate endpoints were adopted as pivotal evidence for accelerated approval of anticancer drugs for more than 30 years. However, concerns regarding clinical evidence quality in trials, particularly in the SATs of anticancer drugs have increasingly been raised. SAT may not always provide strong evidence due to the lack of control and endpoint of overall survival that is typically present in randomized controlled trials.

AREAS COVERED

Clinical trial endpoint adjudication is a crucial factor in surrogate outcome measurement to ensure the data quality of the clinical trial of anticancer drugs. In this review, we systematically discuss the characteristics of adjudications in assessments in surrogate endpoint and safety outcome respectively, which are essential for ensuring reliable and transparent outcomes. Endpoint adjudication effectively reduces potential bias and mitigates variance that may be introduced by investigators when analyzing the medical records for the surrogate endpoints. We analyze the advantages and disadvantages of each type of adjudicator and provide a summary of the roles of adjudicators.

EXPERT OPINION

By suggestion of improving data reliability and transparency in pivotal trials, this review aims to supply a strategy for better clinical investigation for anticancer drugs, ultimately leading to better patient outcomes.

RECIST-Induced Reliable Learning: Geometry-Driven Label Propagation for Universal Lesion Segmentation

Automatic universal lesion segmentation (ULS) from Computed Tomography (CT) images can ease the burden of radiologists and provide a more accurate assessment than the current Response Evaluation Criteria In Solid Tumors (RECIST) guideline measurement. However, this task is underdeveloped due to the absence of large-scale pixel-wise labeled data. This paper presents a weakly-supervised learning framework to utilize the large-scale existing lesion databases in hospital Picture Archiving and Communication Systems (PACS) for ULS. Unlike previous methods to construct pseudo surrogate masks for fully supervised training through shallow interactive segmentation techniques, we propose to unearth the implicit information from RECIST annotations and thus design a unified RECIST-induced reliable learning (RiRL) framework. Particularly, we introduce a novel label generation procedure and an on-the-fly soft label propagation strategy to avoid noisy training and poor generalization problems. The former, named RECIST-induced geometric labeling, uses clinical characteristics of RECIST to preliminarily and reliably propagate the label. With the labeling process, a trimap divides the lesion slices into three regions, including certain foreground, background, and unclear regions, which consequently enables a strong and reliable supervision signal on a wide region. A topological knowledge-driven graph is built to conduct the on-the-fly label propagation for the optimal segmentation boundary to further optimize the segmentation boundary. Experimental results on a public benchmark dataset demonstrate that the proposed method surpasses the SOTA RECIST-based ULS methods by a large margin. Our approach surpasses SOTA approaches over 2.0%, 1.5%, 1.4%, and 1.6% Dice with ResNet101, ResNet50, HRNet, and ResNest50 backbones.

Evaluation of the capability and reproducibility of RECIST 1.1. measurements by technologists in breast cancer follow-up: a pilot study

The evaluation of tumor follow-up according to RECIST 1.1 has become essential in clinical practice given its role in therapeutic decision making. At the same time, radiologists are facing an increase in activity while facing a shortage. Radiographic technologists could contribute to the follow-up of these measures, but no studies have evaluated their ability to perform them. Ninety breast cancer patients were performed three CT follow-ups between September 2017 and August 2021. 270 follow-up treatment CT scans were analyzed including 445 target lesions. The rate of agreement of classifications RECIST 1.1 between five technologists and radiologists yielded moderate (k value between 0.47 and 0.52) and substantial (k value = 0.62 and k = 0.67) agreement values. 112 CT were classified as progressive disease (PD) by the radiologists, and 414 new lesions were identified. The analysis showed a percentage of strict agreement of progressive disease classification between reader-technologists and radiologists ranging from substantial to almost perfect agreement (range 73-97%). Analysis of intra-observer agreement was strong at almost perfect (k > 0.78) for 3 technologists. These results are encouraging regarding the ability of selected technologists to perform measurements according to RECIST 1.1 criteria by CT scan with good identification of disease progression.

Can Radiology Technologists be Trained to Measure Leg Length Discrepancies as Accurately as Pediatric Radiologists?

RATIONALE AND OBJECTIVES

Leg length discrepancy studies are labor intensive. They are procedurally simple and represent inefficient use of the radiologists' time and expertise. We hypothesized that radiology technologists could be trained to measure leg length discrepancies, and that their performance would be statistically equivalent to that of board-certified, fellowship-trained pediatric radiologists.

MATERIAL AND METHODS

Four radiology technologists were selected to participate in a supervised practice session. They independently measured and calculated leg length discrepancies on 10 randomly selected cases. Their performance was compared to measurements obtained by an experienced pediatric radiologist (reference standard). After 1 week, the technologists repeated their measurements on the same cases, which were resorted to simulate new cases. Intraclass correlation coefficients (ICC) determined interobserver agreement between the technologists and radiologist and intra-observer reliability among the technologists.

RESULTS

Among the four technologists, similarity in measurements between session 1 and the reference standard was very high, with ICC values ranging from 0.93 to 0.98 (p < 0.001). The ICC between session 2 and the reference standard was also high, ranging from 0.93 to 0.98 (p < 0.001). Finally, among the four technologists, ICC values between session 1 and session 2 were ≥ 0.96 (p < 0.001).

CONCLUSION

Radiology technologists can be rapidly trained to calculate leg length discrepancies as accurately as a board-certified pediatric radiologist. Delegation of this time-consuming task to technologists or radiology assistants will permit radiologists to spend time on more demanding studies, such as studies that require subspecialty training.

Medical imaging clinical trials unit: A professional need

PURPOSE

To design and describe a management and control tool and the human resources needed to efficiently manage the imaging process within clinical trials for a better quality of care for the patient.

METHODS

A unit was created to efficiently organise the participation of our Medical Imaging Department in clinical trials. This entity was defined and monitored using a customized, flexible and modular software package that provides the necessary information to execute and monitor requests (appointments, protocols, reports, complaints, billing). Various indicators of activity and professional satisfaction were parameterised.

RESULTS

From 2016 to 2020, 367 trials were participated and monitored, 50% of all the hospital clinical trials. The budget of the Medical Imaging Department grew by 47% in this period. The coordination with other departments and principal investigators improved, as shown by surveys (62% fluid and 38% very fluid), with a high perception of collaboration (86%).

CONCLUSIONS

The implementation of a Medical Imaging Clinical Trials Unit involve identifying the tasks, personnel, organisational needs, workflow, monitoring and invoicing. The creation of this Unit has improved the control and traceability of clinical trials within the Department.

Radiology reporting in oncology-oncologists' perspective

BACKGROUND

Structured reporting and standardized criteria are increasingly recognized as means of improving both radiological and clinical practice by allowing for better content and clarity. Our aim was to examine oncologists' opinions and expectations concerning the radiologist's report to identify general needs in daily practice and ways to improve interdisciplinary communication.

METHODS

A 19-question survey was sent to 230 oncologists from three different countries (France, Romania, Switzerland) identified on the online web pages of different hospitals and private clinics. The survey was sent by electronic mail with an online survey program (Google Forms®). All recipients were informed of the purpose of the study. The data were collected by the online survey program and analysed through filtering the results and cross-tabulation.

RESULTS

A total of 52 responses were received (response rate of 22.6%). The majority of the respondents (46/52, 88%) preferred the structured report, which follows a predefined template. Most of the respondents (40/52, 77%) used RECIST 1.1 or iRECIST in tumour assessment. Nearly half of the oncologists (21/52, 40%) measured 1-3 cases per week. On a 10-point Likert scale, 34/52 (65%) oncologists rated their overall level of satisfaction with radiologists' service between 7 and 10. In contrast, 12/52 (19%) oncologists rated the radiologists' service between 1 and 4. Moreover, 42/52 (80%) oncologists acknowledged that reports created by a radiologist with a subspecialty in oncologic imaging were superior to those created by a general radiologist.

CONCLUSION

Structured reports in oncologic patients and the use of RECIST criteria are preferred by oncologists in their daily clinical practice, which signals the need for radiologists also to implement such reports to facilitate communication. Furthermore, most of the oncologists we interviewed recognized the added value provided by radiologists specializing in oncologic imaging. Because this subspecialty is present in only a few countries, generally in large clinics, further training might become a challenge; nevertheless, intensive efforts should be made to enhance expertise in cancer imaging.

A Primer on RECIST 1.1 for Oncologic Imaging in Clinical Drug Trials

Drug discovery and approval in oncology is mediated by the use of imaging to evaluate drug efficacy in clinical trials. Imaging is performed while patients receive therapy to evaluate their response to treatment. Response criteria, specifically Response Evaluation Criteria in Solid Tumors version 1.1 (RECIST 1.1), are standardized and can be used at different time points to classify response into the categories of complete response, partial response, stable disease, or disease progression. At the trial level, categorical responses for all patients are summated into image-based trial endpoints. These outcome measures, including objective response rate (ORR) and progression-free survival (PFS), are characteristics that can be derived from imaging and can be used as surrogates for overall survival (OS). Similar to OS, ORR and PFS describe the efficacy of a drug. U.S. Food and Drug Administration (FDA) regulatory approval requires therapies to demonstrate direct evidence of clinical benefit, such as improved OS. However, multiple programs have been created to expedite drug approval for life-threatening illnesses, including advanced cancer. ORR and PFS have been accepted by the FDA as adequate predictors of OS on which to base drug approval decisions, thus substantially shortening the time and cost of drug development (1). Use of imaging surrogate markers for drug approval has become increasingly common, accounting for more than 90% of approvals through the Accelerated Approval Program and allowing for use of many therapies which have altered the course of cancer. Keywords: Oncology, Tumor Response  RSNA, 2021.

CDISC-compliant clinical trial imaging management system with automatic verification and data Transformation: Focusing on tumor response assessment data in clinical trials

OBJECTIVE

Major issues in imaging data management of tumor response assessment in clinical trials include high human errors in data input and unstandardized data structures, warranting a new breakthrough IT solution. Thus, we aim to develop a Clinical Data Interchange Standards Consortium (CDISC)-compliant clinical trial imaging management system (CTIMS) with automatic verification and transformation modules for implementing the CDISC Study Data Tabulation Model (SDTM) in the tumor response assessment dataset of clinical trials.

MATERIALS AND METHODS

In accordance with various CDISC standards guides and Response Evaluation Criteria in Solid Tumors (RECIST) guidelines, the overall system architecture of CDISC-compliant CTIMS was designed. Modules for standard-compliant electronic case report form (eCRF) to verify data conformance and transform into SDTM data format were developed by experts in diverse fields such as medical informatics, medical, and clinical trial. External validation of the CDISC-compliant CTIMS was performed by comparing it with our previous CTIMS based on real-world data and CDISC validation rules by Pinnacle 21 Community Software.

RESULTS

The architecture of CDISC-compliant CTIMS included the standard-compliant eCRF module of RECIST, the automatic verification module of the input data, and the SDTM transformation module from the eCRF input data to the SDTM datasets based on CDISC Define-XML. This new system was incorporated into our previous CTIMS. External validation demonstrated that all 176 human input errors occurred in the previous CTIMS filtered by a new system yielding zero error and CDISC-compliant dataset. The verified eCRF input data were automatically transformed into the SDTM dataset, which satisfied the CDISC validation rules by Pinnacle 21 Community Software.

CONCLUSIONS

To assure data consistency and high quality of the tumor response assessment data, our new CTIMS can minimize human input error by using standard-compliant eCRF with an automatic verification module and automatically transform the datasets into CDISC SDTM format.

Comparing Paclitaxel-Carboplatin with Paclitaxel-Cisplatin as the Front-Line Chemotherapy for Patients with FIGO IIIC Serous-Type Tubo-Ovarian Cancer

The use of weekly chemotherapy for the treatment of patients with advanced-stage serous-type epithelial Tubo-ovarian cancer (ETOC), and primary peritoneal serous carcinoma (PPSC) is acceptable as the front-line postoperative chemotherapy after primary cytoreductive surgery (PCS). The main component of dose-dense chemotherapy is weekly paclitaxel (80 mg/m²), but it would be interesting to know what is the difference between combination of triweekly cisplatin (20 mg/m²) or triweekly carboplatin (carboplatin area under the curve 5-7 mg/mL per min [AUC 5-7]) in the dose-dense paclitaxel regimen. Therefore, we compared the outcomes of women with Gynecology and Obstetrics (FIGO) stage IIIC ETOC and PPSC treated with PCS and a subsequent combination of dose-dense weekly paclitaxel and triweekly cisplatin (paclitaxel–cisplatin) or triweekly carboplatin using AUC 5 (paclitaxel–carboplatin). Between January 2010 and December 2016, 40 women with International Federation of Gynecology and Obstetrics (FIGO) stage IIIC EOC, FTC, or PPSC were enrolled, including 18 treated with paclitaxel–cisplatin and the remaining 22 treated with paclitaxel–carboplatin. There were no statistically significant differences in disease characteristics of patients between two groups. Outcomes in paclitaxel–cisplatin group seemed to be little better than those in paclitaxel–carboplatin (median progression-free survival [PFS] 30 versus 25 months as well as median overall survival [OS] 58.5 versus 55.0 months); however, neither reached a statistically significant difference. In terms of adverse events (AEs), patients in paclitaxel–carboplatin group had more AEs, with a higher risk of neutropenia and grade 3/4 neutropenia, and the need for a longer period to complete the front-line chemotherapy, and the latter was associated with worse outcome for patients. We found that a period between the first-time chemotherapy to the last dose (6 cycles) of chemotherapy >21 weeks was associated with a worse prognosis in patients compared to that ≤21 weeks, with hazard ratio (HR) of 81.24 for PFS and 9.57 for OS. As predicted, suboptimal debulking surgery (>1 cm) also contributed to a worse outcome than optimal debulking surgery (≤1 cm) with HR of 14.38 for PFS and 11.83 for OS. Based on the aforementioned findings, both regimens were feasible and effective, but maximal efforts should be made to achieve optimal debulking surgery and following the on-schedule administration of dose-dense weekly paclitaxel plus triweekly platinum compounds. Randomized trials validating the findings are warranted.