Multi-reader multi-case studies using the area under the receiver operator characteristic curve as a measure of diagnostic accuracy: systematic review with a focus on quality of data reporting

Deep learning for tubes and lines detection in critical illness: Generalizability and comparison with residents

Background

Artificial intelligence (AI) has been proven useful for the assessment of tubes and lines on chest radiographs of general patients. However, validation on intensive care unit (ICU) patients remains imperative.

Methods

This retrospective case-control study evaluated the performance of deep learning (DL) models for tubes and lines classification on both an external public dataset and a local dataset comprising 303 films randomly sampled from the ICU database. The endotracheal tubes (ETTs), central venous catheters (CVCs), and nasogastric tubes (NGTs) were classified into "Normal," "Abnormal," or "Borderline" positions by DL models with and without rule-based modification. Their performance was evaluated using an experienced radiologist as the standard reference.

Results

The algorithm showed decreased performance on the local ICU dataset, compared to that of the external dataset, decreasing from the Area Under the Curve of Receiver (AUC) of 0.967 (95 % CI 0.965-0.973) to the AUC of 0.70 (95 % CI 0.68-0.77). Significant improvement in the ETT classification task was observed after modifications were made to the model to allow the use of the spatial relationship between line tips and reference anatomy with the improvement of the AUC, increasing from 0.71 (95 % CI 0.70 - 0.75) to 0.86 (95 % CI 0.83 - 0.94).

Conclusions

The externally trained model exhibited limited generalizability on the local ICU dataset. Therefore, evaluating the performance of externally trained AI before integrating it into critical care routine is crucial. Rule-based algorithm may be used in combination with DL to improve results.

G4 & the balanced metric family - a novel approach to solving binary classification problems in medical device validation & verification studies

BACKGROUND

In medical device validation and verification studies, the area under the receiver operating characteristic curve (AUROC) is often used as a primary endpoint despite multiple reports showing its limitations. Hence, researchers are encouraged to consider alternative metrics as primary endpoints. A new metric called G4 is presented, which is the geometric mean of sensitivity, specificity, the positive predictive value, and the negative predictive value. G4 is part of a balanced metric family which includes the Unified Performance Measure (also known as P4) and the Matthews' Correlation Coefficient (MCC). The purpose of this manuscript is to unveil the benefits of using G4 together with the balanced metric family when analyzing the overall performance of binary classifiers.

RESULTS

Simulated datasets encompassing different prevalence rates of the minority class were analyzed under a multi-reader-multi-case study design. In addition, data from an independently published study that tested the performance of a unique ultrasound artificial intelligence algorithm in the context of breast cancer detection was also considered. Within each dataset, AUROC was reported alongside the balanced metric family for comparison. When the dataset prevalence and bias of the minority class approached 50%, all three balanced metrics provided equivalent interpretations of an AI's performance. As the prevalence rate increased / decreased and the data became more imbalanced, AUROC tended to overvalue / undervalue the true classifier performance, while the balanced metric family was resistant to such imbalance. Under certain circumstances where data imbalance was strong (minority-class prevalence < 10%), MCC was preferred for standalone assessments while P4 provided a stronger effect size when evaluating between-groups analyses. G4 acted as a middle ground for maximizing both standalone assessments and between-groups analyses.

CONCLUSIONS

Use of AUROC as the primary endpoint in binary classification problems provides misleading results as the dataset becomes more imbalanced. This is explicitly noticed when incorporating AUROC in medical device validation and verification studies. G4, P4, and MCC do not share this limitation and paint a more complete picture of a medical device's performance in a clinical setting. Therefore, researchers are encouraged to explore the balanced metric family when evaluating binary classification problems.

Impact of Patient-reported Symptom Information on the Interpretation of MRI of the Lumbar Spine

Background Distinguishing lumbar pain generators from incidental findings at MRI can be difficult. Dictated reports may become lists of findings that cannot be ranked in order of diagnostic importance. Purpose To determine whether patient-reported symptom information can improve the interpretation of lumbar spine MRI by using the spine specialist as the reference standard. Materials and Methods This prospective, single-center, multireader study analyzed 240 participants who completed pre-MRI symptom questionnaires between May 2022 and February 2023. At the time of clinical MRI reporting, radiologists recorded pain generators in consecutive participants, creating two study groups by alternating interpretations with versus without symptom questionnaire results (SQR). Diagnostic certainty was recorded using a numeric scale of 0 to 100. Types, levels, and sides of pain generators were compared with reference diagnoses by calculating Cohen κ values with 95% CIs. Participant characteristics and diagnostic certainties were compared using the Wilcoxon rank sum, Pearson χ2, or Kruskal-Wallis test. Interrater agreement was analyzed. Results There was no difference in age (P = .69) or sex (P = .60) between participants using SQR (n = 120; mean age, 61.0 years; 62 female) and not using SQR (n = 120; mean age, 62.5 years; 67 female). When radiologists were compared with specialists, agreements on pain generators were almost perfect for interpretations using SQR (type: κ = 0.82 [95% CI: 0.74,0.89]; level: κ = 0.88 [95% CI: 0.80, 0.95]; side: κ = 0.84 [95% CI: 0.75, 0.92]), but only fair to moderate for interpretations not using SQR (type: κ = 0.26 [95% CI: 0.15, 0.36]; level: κ = 0.51 [95% CI: 0.39, 0.63]; side: κ = 0.30 [95% CI: 0.18, 0.42]) (all P < .001). Diagnostic certainty was higher for MRI interpretations using SQR (mean, 80.4 ± 14.9 [SD]) than MRI interpretations not using SQR (60.5 ± 17.7) (P < .001). Interrater agreements were substantial (κ = 0.65-0.78) for MRI interpretations using SQR but only fair to moderate (κ = 0.24-0.49) for MRI interpretations not using SQR (all P < .001). Conclusion Patient-reported symptom information enabled radiologists to achieve nearly perfect diagnostic agreement with clinical experts. © RSNA, 2024 See also the editorial by Isikbay and Shah in this issue.

Evaluation of the impact of artificial intelligence-assisted image interpretation on the diagnostic performance of clinicians in identifying pneumothoraces on plain chest X-ray: a multi-case multi-reader study

BACKGROUND

Artificial intelligence (AI)-assisted image interpretation is a fast-developing area of clinical innovation. Most research to date has focused on the performance of AI-assisted algorithms in comparison with that of radiologists rather than evaluating the algorithms' impact on the clinicians who often undertake initial image interpretation in routine clinical practice. This study assessed the impact of AI-assisted image interpretation on the diagnostic performance of frontline acute care clinicians for the detection of pneumothoraces (PTX).

METHODS

A multicentre blinded multi-case multi-reader study was conducted between October 2021 and January 2022. The online study recruited 18 clinician readers from six different clinical specialties, with differing levels of seniority, across four English hospitals. The study included 395 plain CXR images, 189 positive for PTX and 206 negative. The reference standard was the consensus opinion of two thoracic radiologists with a third acting as arbitrator. General Electric Healthcare Critical Care Suite (GEHC CCS) PTX algorithm was applied to the final dataset. Readers individually interpreted the dataset without AI assistance, recording the presence or absence of a PTX and a confidence rating. Following a 'washout' period, this process was repeated including the AI output.

RESULTS

Analysis of the performance of the algorithm for detecting or ruling out a PTX revealed an overall AUROC of 0.939. Overall reader sensitivity increased by 11.4% (95% CI 4.8, 18.0, p=0.002) from 66.8% (95% CI 57.3, 76.2) unaided to 78.1% aided (95% CI 72.2, 84.0, p=0.002), specificity 93.9% (95% CI 90.9, 97.0) without AI to 95.8% (95% CI 93.7, 97.9, p=0.247). The junior reader subgroup showed the largest improvement at 21.7% (95% CI 10.9, 32.6), increasing from 56.0% (95% CI 37.7, 74.3) to 77.7% (95% CI 65.8, 89.7, p<0.01).

CONCLUSION

The study indicates that AI-assisted image interpretation significantly enhances the diagnostic accuracy of clinicians in detecting PTX, particularly benefiting less experienced practitioners. While overall interpretation time remained unchanged, the use of AI improved diagnostic confidence and sensitivity, especially among junior clinicians. These findings underscore the potential of AI to support less skilled clinicians in acute care settings.

A Multireader Multicase (MRMC) Receiver Operating Characteristic (ROC) Study Evaluating Noninferiority of Quantitative Transmission (QT) Ultrasound to Digital Breast Tomosynthesis (DBT) on Detection and Recall of Breast Lesions

RATIONALE AND OBJECTIVES

Quantitative transmission (QT) imaging is an emerging volumetric ultrasound modality for women too young for mammography. QT images tissue without overlap seen in mammography, thereby can potentially improve breast mass detection and characterization and noncancer recall. We compared radiologists' interpretation of QT vs digital breast tomosynthesis (DBT) with a multireader multicase observer performance study.

MATERIALS AND METHODS

Study subjects received screening DBT and QT scans in HIPAA-compliant, institutional review board-approved prospective case-collection studies at four clinical sites. Twenty-four Mammography Quality Standards Act-qualified radiologists interpreted 177 cases (66 with cancer, atypia, or solid mass and 111 normal or with nonsolid benign abnormality), first QT, then 2 weeks later DBT synthesized 2D-views. Readers reported up to three findings per case and for each finding a recall or no recall decision and confidence of that decision. The study hypothesis was area under receiver operating characteristic curve (AUC) of QT was noninferior to DBT. Sensitivity and specificity were also compared.

RESULTS

AUC of QT (0.746 ± 0.028, mean ± SD) was noninferior to DBT (0.700 ± 0.028) for AUC difference margin of -0.05 (P < .05). AUC difference was 0.046 ± 0.028 (95% CI: [-0.008, 0.101]). Sensitivity was 70.6 ± 7.2% for QT and 85.2 ± 6.4% for DBT, specificity was 60.1 ± 12.3% vs 37.2 ± 11.0%, and both differences were statistically significant. Of a total of 21 cases of cysts, readers recommended recall, on average, in 1.1 ± 1.4 cases with QT, but not with DBT, and 10.6 ± 2.2 cases with DBT, but not with QT.

CONCLUSION

QT can be a potential alternative to mammography for breast cancer screening of women too young to undergo mammography.

Artificial intelligence-assisted diagnosis of congenital heart disease and associated pulmonary arterial hypertension from chest radiographs: A multi-reader multi-case study

OBJECTIVES

To explore the possibility of automatic diagnosis of congenital heart disease (CHD) and pulmonary arterial hypertension associated with CHD (PAH-CHD) from chest radiographs using artificial intelligence (AI) technology and to evaluate whether AI assistance could improve clinical diagnostic accuracy.

MATERIALS AND METHODS

A total of 3255 frontal preoperative chest radiographs (1174 CHD of any type and 2081 non-CHD) were retrospectively obtained. In this study, we adopted ResNet18 pretrained with the ImageNet database to establish diagnostic models. Radiologists diagnosed CHD/PAH-CHD from 330/165 chest radiographs twice: the first time, 50% of the images were accompanied by AI-based classification; after a month, the remaining 50% were accompanied by AI-based classification. Diagnostic results were compared between the radiologists and AI models, and between radiologists with and without AI assistance.

RESULTS

The AI model achieved an average area under the receiver operating characteristic curve (AUC) of 0.948 (sensitivity: 0.970, specificity: 0.982) for CHD diagnoses and an AUC of 0.778 (sensitivity: 0.632, specificity: 0.925) for identifying PAH-CHD. In the 330 balanced (165 CHD and 165 non-CHD) testing set, AI achieved higher AUCs than all 5 radiologists in the identification of CHD (0.670-0.858) and PAH-CHD (0.610-0.688). With AI assistance, the mean ± standard error AUC of radiologists was significantly improved for CHD (ΔAUC + 0.096, 95 % CI: 0.001-0.190; P = 0.048) and PAH-CHD (ΔAUC + 0.066, 95 % CI: 0.010-0.122; P = 0.031) diagnosis.

CONCLUSION

Chest radiograph-based AI models can detect CHD and PAH-CHD automatically. AI assistance improved radiologists' diagnostic accuracy, which may facilitate a timely initial diagnosis of CHD and PAH-CHD.

Artificial Intelligence Applications in Cardiovascular Magnetic Resonance Imaging: Are We on the Path to Avoiding the Administration of Contrast Media?

In recent years, cardiovascular imaging examinations have experienced exponential growth due to technological innovation, and this trend is consistent with the most recent chest pain guidelines. Contrast media have a crucial role in cardiovascular magnetic resonance (CMR) imaging, allowing for more precise characterization of different cardiovascular diseases. However, contrast media have contraindications and side effects that limit their clinical application in determinant patients. The application of artificial intelligence (AI)-based techniques to CMR imaging has led to the development of non-contrast models. These AI models utilize non-contrast imaging data, either independently or in combination with clinical and demographic data, as input to generate diagnostic or prognostic algorithms. In this review, we provide an overview of the main concepts pertaining to AI, review the existing literature on non-contrast AI models in CMR, and finally, discuss the strengths and limitations of these AI models and their possible future development.

Diagnostic performance of deep learning-based vessel extraction and stenosis detection on coronary computed tomography angiography for coronary artery disease: a multi-reader multi-case study

BACKGROUND

Post-processing and interpretation of coronary CT angiography (CCTA) imaging are time-consuming and dependent on the reader's experience. An automated deep learning (DL)-based imaging reconstruction and diagnosis system was developed to improve diagnostic accuracy and efficiency.

METHODS

Our study including 374 cases from five sites, inviting 12 radiologists, assessed the DL-based system in diagnosing obstructive coronary disease with regard to diagnostic performance, imaging post-processing and reporting time of radiologists, with invasive coronary angiography as a standard reference. The diagnostic performance of DL system and DL-assisted human readers was compared with the traditional method of human readers without DL system.

RESULTS

Comparing the diagnostic performance of human readers without DL system versus with DL system, the AUC was improved from 0.81 to 0.82 (p < 0.05) at patient level and from 0.79 to 0.81 (p < 0.05) at vessel level. An increase in AUC was observed in inexperienced radiologists (p < 0.05), but was absent in experienced radiologists. Regarding diagnostic efficiency, comparing the DL system versus human reader, the average post-processing and reporting time was decreased from 798.60 s to 189.12 s (p < 0.05). The sensitivity and specificity of using DL system alone were 93.55% and 59.57% at patient level and 83.23% and 79.97% at vessel level, respectively.

CONCLUSIONS

With the DL system serving as a concurrent reader, the overall post-processing and reading time was substantially reduced. The diagnostic accuracy of human readers, especially for inexperienced readers, was improved. DL-assisted human reader had the potential of being the reading mode of choice in clinical routine.

Deep Learning System Boosts Radiologist Detection of Intracranial Hemorrhage

BACKGROUND

Intracranial hemorrhage (ICH) requires emergent medical treatment for positive outcomes. While previous artificial intelligence (AI) solutions achieved rapid diagnostics, none were shown to improve the performance of radiologists in detecting ICHs. Here, we show that the Caire ICH artificial intelligence system enhances a radiologist's ICH diagnosis performance.

METHODS

A dataset of non-contrast-enhanced axial cranial computed tomography (CT) scans (n=532) were labeled for the presence or absence of an ICH. If an ICH was detected, its ICH subtype was identified. After a washout period, the three radiologists reviewed the same dataset with the assistance of the Caire ICH system. Performance was measured with respect to reader agreement, accuracy, sensitivity, and specificity when compared to the ground truth, defined as reader consensus.

RESULTS

Caire ICH improved the inter-reader agreement on average by 5.76% in a dataset with an ICH prevalence of 74.3%. Further, radiologists using Caire ICH detected an average of 18 more ICHs and significantly increased their accuracy by 6.15%, their sensitivity by 4.6%, and their specificity by 10.62%. The Caire ICH system also improved the radiologist's ability to accurately identify the ICH subtypes present.

CONCLUSION

The Caire ICH device significantly improves the performance of a cohort of radiologists. Such a device has the potential to be a tool that can improve patient outcomes and reduce misdiagnosis of ICH.

Assessing Detection Accuracy of Computerized Sonographic Features and Computer-Assisted Reading Performance in Differentiating Thyroid Cancers

For ultrasound imaging of thyroid nodules, medical guidelines are all based on findings of sonographic features to provide clinicians management recommendations. Due to the recent development of artificial intelligence and machine learning (AI/ML) technologies, there have been computer-assisted detection (CAD) software devices available for clinical use to detect and quantify the sonographic features of thyroid nodules. This study is to validate the accuracy of the computerized sonographic features (CSF) by a CAD software device, namely, AmCAD-UT, and then to assess how the reading performance of clinicians (readers) can be improved providing the computerized features. The feature detection accuracy is tested against the ground truth established by a panel of thyroid specialists and a multiple-reader multiple-case (MRMC) study is performed to assess the sequential reading performance with the assistance of the CSF. Five computerized features, including anechoic area, hyperechoic foci, hypoechoic pattern, heterogeneous texture, and indistinct margin, were tested, with AUCs ranging from 0.888~0.946, 0.825~0.913, 0.812~0.847, 0.627~0.77, and 0.676~0.766, respectively. With the five CSFs, the sequential reading performance of 18 clinicians is found significantly improved, with the AUC increasing from 0.720 without CSF to 0.776 with CSF. Our studies show that the computerized features are consistent with the clinicians’ findings and provide additional value in assisting sonographic diagnosis.

Multireader Diagnostic Accuracy Imaging Studies: Fundamentals of Design and Analysis

The design and analysis of multireader multicase (MRMC) studies are quite challenging. These studies differ from most medical studies because they need a reference standard and sampling from two populations (ie, reader and patient populations). They are quite expensive to conduct, requiring a good deal of readers' time for image interpretation. One common problem is the use of imperfect reference standards, often correlated with the test or tests being evaluated. Another common issue is oversimplification of the multidimensional MRMC data. In this study, the fundamentals of MRMC study design and analysis are reviewed. The goal is to provide investigators with a guide to the fundamentals of MRMC design and analysis, with references to more detailed discussions. In addition, readers are updated on newer areas of research, including correction for studies with multiple diagnostic accuracy end points and adjustment for location bias.

A review of explainable and interpretable AI with applications in COVID-19 imaging

The development of medical imaging artificial intelligence (AI) systems for evaluating COVID-19 patients has demonstrated potential for improving clinical decision making and assessing patient outcomes during the recent COVID-19 pandemic. These have been applied to many medical imaging tasks, including disease diagnosis and patient prognosis, as well as augmented other clinical measurements to better inform treatment decisions. Because these systems are used in life-or-death decisions, clinical implementation relies on user trust in the AI output. This has caused many developers to utilize explainability techniques in an attempt to help a user understand when an AI algorithm is likely to succeed as well as which cases may be problematic for automatic assessment, thus increasing the potential for rapid clinical translation. AI application to COVID-19 has been marred with controversy recently. This review discusses several aspects of explainable and interpretable AI as it pertains to the evaluation of COVID-19 disease and it can restore trust in AI application to this disease. This includes the identification of common tasks that are relevant to explainable medical imaging AI, an overview of several modern approaches for producing explainable output as appropriate for a given imaging scenario, a discussion of how to evaluate explainable AI, and recommendations for best practices in explainable/interpretable AI implementation. This review will allow developers of AI systems for COVID-19 to quickly understand the basics of several explainable AI techniques and assist in the selection of an approach that is both appropriate and effective for a given scenario.

Automated Echocardiographic Detection of Severe Coronary Artery Disease Using Artificial Intelligence

OBJECTIVES

The purpose of this study was to establish whether an artificially intelligent (AI) system can be developed to automate stress echocardiography analysis and support clinician interpretation.

BACKGROUND

Coronary artery disease is the leading global cause of mortality and morbidity and stress echocardiography remains one of the most commonly used diagnostic imaging tests.

METHODS

An automated image processing pipeline was developed to extract novel geometric and kinematic features from stress echocardiograms collected as part of a large, United Kingdom-based prospective, multicenter, multivendor study. An ensemble machine learning classifier was trained, using the extracted features, to identify patients with severe coronary artery disease on invasive coronary angiography. The model was tested in an independent U.S.

STUDY

How availability of an AI classification might impact clinical interpretation of stress echocardiograms was evaluated in a randomized crossover reader study.

RESULTS

Acceptable classification accuracy for identification of patients with severe coronary artery disease in the training data set was achieved on cross-fold validation based on 31 unique geometric and kinematic features, with a specificity of 92.7% and a sensitivity of 84.4%. This accuracy was maintained in the independent validation data set. The use of the AI classification tool by clinicians increased inter-reader agreement and confidence as well as sensitivity for detection of disease by 10% to achieve an area under the receiver-operating characteristic curve of 0.93.

CONCLUSION

Automated analysis of stress echocardiograms is possible using AI and provision of automated classifications to clinicians when reading stress echocardiograms could improve accuracy, inter-reader agreement, and reader confidence.

Effect of a comprehensive deep-learning model on the accuracy of chest x-ray interpretation by radiologists: a retrospective, multireader multicase study

BACKGROUND

Chest x-rays are widely used in clinical practice; however, interpretation can be hindered by human error and a lack of experienced thoracic radiologists. Deep learning has the potential to improve the accuracy of chest x-ray interpretation. We therefore aimed to assess the accuracy of radiologists with and without the assistance of a deep-learning model.

METHODS

In this retrospective study, a deep-learning model was trained on 821 681 images (284 649 patients) from five data sets from Australia, Europe, and the USA. 2568 enriched chest x-ray cases from adult patients (≥16 years) who had at least one frontal chest x-ray were included in the test dataset; cases were representative of inpatient, outpatient, and emergency settings. 20 radiologists reviewed cases with and without the assistance of the deep-learning model with a 3-month washout period. We assessed the change in accuracy of chest x-ray interpretation across 127 clinical findings when the deep-learning model was used as a decision support by calculating area under the receiver operating characteristic curve (AUC) for each radiologist with and without the deep-learning model. We also compared AUCs for the model alone with those of unassisted radiologists. If the lower bound of the adjusted 95% CI of the difference in AUC between the model and the unassisted radiologists was more than -0·05, the model was considered to be non-inferior for that finding. If the lower bound exceeded 0, the model was considered to be superior.

FINDINGS

Unassisted radiologists had a macroaveraged AUC of 0·713 (95% CI 0·645-0·785) across the 127 clinical findings, compared with 0·808 (0·763-0·839) when assisted by the model. The deep-learning model statistically significantly improved the classification accuracy of radiologists for 102 (80%) of 127 clinical findings, was statistically non-inferior for 19 (15%) findings, and no findings showed a decrease in accuracy when radiologists used the deep-learning model. Unassisted radiologists had a macroaveraged mean AUC of 0·713 (0·645-0·785) across all findings, compared with 0·957 (0·954-0·959) for the model alone. Model classification alone was significantly more accurate than unassisted radiologists for 117 (94%) of 124 clinical findings predicted by the model and was non-inferior to unassisted radiologists for all other clinical findings.

INTERPRETATION

This study shows the potential of a comprehensive deep-learning model to improve chest x-ray interpretation across a large breadth of clinical practice.

FUNDING

Annalise.ai.

Evaluation of Lower-Dose Spiral Head CT for Detection of Intracranial Findings Causing Neurologic Deficits

BACKGROUND AND PURPOSE

Despite the frequent use of unenhanced head CT for the detection of acute neurologic deficit, the radiation dose for this exam varies widely. Our aim was to evaluate the performance of lower-dose head CT for detection of intracranial findings resulting in acute neurologic deficit.

MATERIALS AND METHODS

Projection data from 83 patients undergoing unenhanced spiral head CT for suspected neurologic deficits were collected. Cases positive for infarction, intra-axial hemorrhage, mass, or extra-axial hemorrhage required confirmation by histopathology, surgery, progression of findings, or corresponding neurologic deficit; cases negative for these target diagnoses required negative assessments by two neuroradiologists and a clinical neurologist. A routine dose head CT was obtained using 250 effective mAs and iterative reconstruction. Lower-dose configurations were reconstructed (25-effective mAs iterative reconstruction, 50-effective mAs filtered back-projection and iterative reconstruction, 100-effective mAs filtered back-projection and iterative reconstruction, 200-effective mAs filtered back-projection). Three neuroradiologists circled findings, indicating diagnosis, confidence (0-100), and image quality. The difference between the jackknife alternative free-response receiver operating characteristic figure of merit at routine and lower-dose configurations was estimated. A lower 95% CI estimate of the difference greater than -0.10 indicated noninferiority.

RESULTS

Forty-two of 83 patients had 70 intracranial findings (29 infarcts, 25 masses, 10 extra- and 6 intra-axial hemorrhages) at routine head CT (CT dose index = 38.3 mGy). The routine-dose jackknife alternative free-response receiver operating characteristic figure of merit was 0.87 (95% CI, 0.81-0.93). Noninferiority was shown for 100-effective mAs iterative reconstruction (figure of merit difference, -0.04; 95% CI, -0.08 to 0.004) and 200-effective mAs filtered back-projection (-0.02; 95% CI, -0.06 to 0.02) but not for 100-effective mAs filtered back-projection (-0.06; 95% CI, -0.10 to -0.02) or lower-dose levels. Image quality was better at higher-dose levels and with iterative reconstruction (P < .05).

CONCLUSIONS

Observer performance for dose levels using 100-200 eff mAs was noninferior to that observed at 250 effective mAs with iterative reconstruction, with iterative reconstruction preserving noninferiority at a mean CT dose index of 15.2 mGy.

Impact of Deep Learning Assistance on the Histopathologic Review of Lymph Nodes for Metastatic Breast Cancer

Supplemental Digital Content is available in the text.

Advances in the quality of whole-slide images have set the stage for the clinical use of digital images in anatomic pathology. Along with advances in computer image analysis, this raises the possibility for computer-assisted diagnostics in pathology to improve histopathologic interpretation and clinical care. To evaluate the potential impact of digital assistance on interpretation of digitized slides, we conducted a multireader multicase study utilizing our deep learning algorithm for the detection of breast cancer metastasis in lymph nodes. Six pathologists reviewed 70 digitized slides from lymph node sections in 2 reader modes, unassisted and assisted, with a wash-out period between sessions. In the assisted mode, the deep learning algorithm was used to identify and outline regions with high likelihood of containing tumor. Algorithm-assisted pathologists demonstrated higher accuracy than either the algorithm or the pathologist alone. In particular, algorithm assistance significantly increased the sensitivity of detection for micrometastases (91% vs. 83%, P=0.02). In addition, average review time per image was significantly shorter with assistance than without assistance for both micrometastases (61 vs. 116 s, P=0.002) and negative images (111 vs. 137 s, P=0.018). Lastly, pathologists were asked to provide a numeric score regarding the difficulty of each image classification. On the basis of this score, pathologists considered the image review of micrometastases to be significantly easier when interpreted with assistance (P=0.0005). Utilizing a proof of concept assistant tool, this study demonstrates the potential of a deep learning algorithm to improve pathologist accuracy and efficiency in a digital pathology workflow.

Clinical relevance of different handgrip strength indexes and cardiovascular disease risk factors: A cross-sectional study in suburb-dwelling elderly Chinese

BACKGROUND

Reduced muscle strength, as measured by handgrip strength (HS), has been associated with an increased risk of cardiovascular disease (CVD). The aim of this study was to examine the association between different HS indexes and CVD risk factors in elderly Chinese individuals. We also determine optimal cutoffs of HS indexes for predicting CVD risk factors.

METHODS

Data were obtained from 603 men and 789 women aged ≥60 years (average age 66.8 ± 6.4 y). These study participants were recruited in the suburb area of Tianjin, China. An individual was considered a patient when they exhibited any one of three CVD risk factors: diabetes mellitus, hypertension and dyslipidemia. All participants were interviewed face-to-face. In addition, serum samples were collected from all participants, and all participants underwent measures of anthropometry and HS.

RESULTS

The optimal cutoffs were 0.376 of HS/weight in men and 0.726 of HS/body fat mass in women for predicting diabetes mellitus. The adjusted odds ratios (ORs) of at least one CVD risk factor for those with low muscle strength identified by HS/body fat mass were 2.14 (95% confidence interval [CI]: 1.53, 3.44; p < 0.001) in men and 2.32 (95% CI: 1.60, 3.29; p < 0.001) in women.

CONCLUSION

HS/body fat mass appear to be the index best associated with CVD risk factors except diabetes mellitus in men. The optimal cutoffs of HS indexes have the potential to identify elderly adults at risk of CVD.

Integration of a Zero-footprint Cloud-based Picture Archiving and Communication System with Customizable Forms for Radiology Research and Education

RATIONALE AND OBJECTIVES

The purpose of this study was to integrate web-based forms with a zero-footprint cloud-based Picture Archiving and Communication Systems (PACS) to create a tool of potential benefit to radiology research and education.

MATERIALS AND METHODS

Web-based forms were created with a front-end and back-end architecture utilizing common programming languages including Vue.js, Node.js and MongoDB, and integrated into an existing zero-footprint cloud-based PACS.

RESULTS

The web-based forms application can be accessed in any modern internet browser on desktop or mobile devices and allows the creation of customizable forms consisting of a variety of questions types. Each form can be linked to an individual DICOM examination or a collection of DICOM examinations.

CONCLUSIONS

Several uses are demonstrated through a series of case studies, including implementation of a research platform for multi-reader multi-case (MRMC) studies and other imaging research, and creation of an online Objective Structure Clinical Examination (OSCE) and an educational case file.

Awareness and Knowledge about Occupational Therapy in Jordan

Knowledge and awareness about occupational therapy (OT) are essential for the delivery of quality care to all clients and for occupational therapists' (OTRs) job satisfaction. OT has been a poorly understood profession in Jordan. The current study reports on the assessment of Jordanians' awareness and knowledge of occupational therapy. Convenience sampling was used. There were 829 participants (474 males, 355 females), with mean age of 32 ± 11.6 yrs. They were recruited from the three main geographical areas of Jordan (northern, central, and southern) and from all educational levels. The sample included 222 (26.8%) healthcare personnel, 146 (17.6%) clients, and 461 (55.6%) lay persons. Participants completed questionnaires, and the results revealed that 48% of the sample had poor or no knowledge about OT, while 28.3% were unaware of it. Also, OT was commonly (50%) perceived to be exclusively targeting people with disabilities (PWDs) and neurological and physical conditions (58% and 53%, resp.) in addition to exclusively providing services for the rehabilitation of the upper extremity (48%). Common misconceptions associated with OT were that OTRs prescribe medication (43%) and OTRs are physiotherapists (44%). These preliminary findings suggest that efforts need to be directed by OTRs, the Jordanian Society of Occupational Therapy (JSOT), and the Ministry of Health to preserve the OT identity and value and promote knowledge about OT in the public and among members of interdisciplinary teams. More interprofessional learning needs to be incorporated within the curricula and placements of all healthcare personnels.

How Does the Display Luminance Level Affect Detectability of Breast Microcalcifications and Spiculated Lesions in Digital Breast Tomosynthesis (DBT) Images?

RATIONALE AND OBJECTIVES

This study evaluates the influence of the calibrated luminance level of medical displays in the detectability of microcalcifications and spiculated lesions in digital breast tomosynthesis images.

MATERIALS AND METHODS

Four models of medical displays with calibrated maximum and minimum luminance, respectively, ranging from 500 to 1000 cd/m² and from 0.5 to 1.0 cd/m², were investigated. Forty-eight studies were selected by a senior radiologist: 16 with microcalcifications, 16 with spiculated lesions, and 16 without lesions. All images were anonymized and blindly evaluated by one senior and two junior radiologists. For each study, lesion presence or absence and localization statements, interpretative difficulty level, and overall quality were reported. Cohen's kappa statistic was computed between monitors and within or between radiologists to estimate the reproducibility in correctly identifying lesions; for multireader-multicase analysis, the weighted jackknife alternative free-response receiver operating characteristic statistical tool was applied.

RESULTS

Intraradiologist reproducibility ranged from 0.75 to 1.00. Interreader as well as reader-truth agreement values were >0.80 and higher with the two 1000 cd/m² luminance displays than with the lower luminance displays for each radiologist. Performances in the detectability of breast lesions were significantly greater with the 1000 cd/m² luminance displays when compared to the display with the lowest luminance value (P value <0.001).

CONCLUSIONS

Our findings highlight the role of display luminance level on the accuracy of detecting breast lesions.