Reliability and validity of a mobile phone for radiographic assessment of ankle injuries: a randomized inter- and intraobserver agreement study

Reliability of Assessing Distal Radius Fractures on a Smartphone Versus PACS

BACKGROUND

Communication between health care providers is becoming more intertwined with technology. During the pandemic, telehealth strategies grew exponentially. Remote viewing of imaging on a smartphone may offer efficient communication; however, the reliability of injury assessment when compared with traditional methods is not known. The purpose of this study was to evaluate intraobserver and interobserver reliability of distal radius fracture radiograph review for smartphone versus traditional Picture Archiving and Communication System (PACS).

METHODS

Eight evaluators (3 attending hand surgeons, 3 hand surgery fellows, 2 orthopedic residents) evaluated 26 distal radius fracture radiographs on 2 different viewers: smartphone or PACS. The reviewers were asked to record: (1) operative or nonoperative preference; (2) fracture classification (based on Fernandez and Jupiter); and (3) treatment strategy (volar plate, dorsal plate, pins, cast, bridge plate, or fragment-specific fixation). The percentage of intraobserver agreement was recorded for each observer. Reliability was calculated using Fleiss' kappa coefficient for intraobserver and interobserver agreement and graded by strength of correlation.

RESULTS

Intraobserver agreement averaged 97% when deciding between operative and nonoperative treatment, 76% for classification, and 84% for treatment. Kappa scores were graded as "excellent" for operative decision and "substantial" for classification and treatment. Attendings and fellows generally had higher agreement than that of residents. Interobserver agreement was graded as "substantial" for all categories for both PACS and smartphone.

CONCLUSIONS

Evaluation of radiographs on a smartphone for the purpose of treating distal radius fractures does not appear to be significantly different from an evaluation on traditional PACS.

Diagnostic accuracy and reliability of smartphone captured radiologic images communicated via WhatsApp®

INTRODUCTION

Sending radiographic images as instant messages have become a common means of communication between physicians, aiding in triaging and transfer decision-making in emergencies. While use of technology is increasing, this is not the case for the underserved or rural areas of South Africa with no picture archiving and communications system or advanced hardware in place. In these areas, the medical staffing population have nearly universal access to smartphones and could benefit from the ability to share images quickly and easily with trained radiologists. South African data on diagnostic reliability of smartphone captured radiology images is lacking. The objective of the study was to determine the accuracy and reliability of diagnoses made on radiologic images captured with smartphone compared to radiologic images on picture archiving and communication system(PACS).

METHODS

A cross-sectional study was conducted with radiographs from June 2018 to July 2019 selected from the PACS system at Pelonomi Tertiary Hospital. Images were displayed on PACS computer screen and captured by principal researcher using a smartphone. Five radiology registrars received the images via WhatsApp® and reviewed them on smartphones. After three weeks, registrars viewed images in random order on PACS stations. McNemar's test was used to compare the diagnostic accuracy of smartphone vs PACS and Kappa values calculated for agreement. Reliability was assessed by analysing the results of different registrars and diagnoses separately.

RESULTS

135 X-rays, representative of common emergency conditions, were selected. For all registrars, PACS accuracy was generally higher than smartphone accuracy. The Kappa values all indicated fair to moderate agreement between smartphone and PACS diagnosis.

CONCLUSION

Capturing radiographic images using at least 12-megapixel smartphone and sharing them via WhatsApp® is a reliable method that can be used with a high degree of confidence in emergencies to aid clinical decision making. This method of viewing medical imaging is however not a substitution for images viewed on PACS.

The role of medical smartphone apps in clinical decision-support: A literature review

INTRODUCTION

The now ubiquitous smartphone has huge potential to assist clinical decision-making across the globe. However, the rapid pace of digitalisation contrasts starkly with the slower rate of medical research and publication. This review explores the evidence base that exists to validate and evaluate the use of medical decision-support apps. The resultant findings will inform appropriate and pragmatic evaluation strategies for future clinical app developers and provide a scientific and cultural context for research priorities in this field.

METHOD

Medline, Embase and Cochrane databases were searched for clinical trials concerning decision support and smart phones from 2007 (introduction of first smartphone iPhone) until January 2019.

RESULTS

Following exclusions, 48 trials and one Cochrane review were included for final analysis. Whilst diagnostic accuracy studies are plentiful, clinical trials are scarce. App research methodology was further interrogated according to setting and decision-support modality: e.g. camera-based, guideline-based, predictive models. Description of app development pathways and regulation were highly varied. Global health emerged as an early adopter of decision-support apps and this field is leading implementation and evaluation.

CONCLUSION

Clinical decision-support apps have considerable potential to enhance access to care and quality of care, but the medical community must rise to the challenge of modernising its approach if it is truly committed to capitalising on the opportunities of digitalisation.

Personal smartphones for neonatal diagnostic imaging: A prospective crossover study

AIM

Neonatal clinicians may be asked to review X-ray images when unable to directly access the original image. Transmitting an X-ray image to a smartphone is a technique increasingly being used by clinicians in a number of settings. Minimal data exist on its use in the neonatal setting. Our aim was to compare the ability of neonatal clinicians to correctly diagnose a pneumothorax from a chest radiograph (CXR) viewed on a smartphone, paralleled with the same image viewed on a computer screen.

METHODS

We investigated the accuracy, sensitivity, specificity, positive predictive value and negative predicative value of two methods of viewing CXR images to diagnose pneumothorax, compared with a gold standard radiologist report. Clinicians were presented with 40 CXR images on two occasions, at least 1 week apart. Images were viewed once on a smartphone and once on a computer screen. Both the viewing method and viewing order of the images were randomised. The clinical details of the infant at the time the CXR was taken were provided, and participants were asked if a pneumothorax was present.

RESULTS

Twenty-one clinicians viewed all CXR images using both viewing methods (840 paired observations). There was no difference in accuracy of detecting pneumothorax between viewing methods, 81% correctly identified a pneumothorax using the smartphone, versus 80% using the computer screen (P = 0.40 (95% CI), difference -4 to 1.5%).

CONCLUSION

Diagnosis of neonatal pneumothorax was as accurate using a smartphone as viewing CXR images displayed on a computer screen when clinicians were presented with X-rays with diagnosis of pneumothorax.

Use of digital images to aid in the decision-making for acute upper extremity trauma referral

This study evaluated the use of digital smartphone images in the decision-making for acute upper extremity trauma referrals. Surgeons (n = 15) were presented with ten upper limb trauma scenarios for consideration of immediate transfer. Based on verbal history and with additional images, participants were asked questions regarding diagnosis, injured tissues, recommended management and diagnostic and treatment confidence. Statistical analyses evaluated confidence level changes and relationships between confidence levels and independent variables. Confidence levels for diagnosis and treatment were increased with the provision of smartphone images, and this was statistically significant. The decision to transfer was changed in 22%. The photographs were more useful for amputation versus non-amputation injuries (diagnosis and treatment) and hand versus forearm injuries (diagnosis), and these differences reached statistical significance. Smartphone digital images were shown to be useful for decision-making in acute upper extremity trauma referrals. This improved communication may have implications for health cost savings and patient burden by minimizing unnecessary acute transfers.

LEVEL OF EVIDENCE

Diagnostic Level III.

Comparison of diagnostic accuracy of plain film radiographs between original film and smartphone capture: a pilot study

The use of mobile devices for medical image capture has become increasingly popular given the widespread use of smartphone cameras. Prior studies have generally compared mobile phone capture images to digitized images. However, many underserved and rural areas without picture archiving and communication systems (PACS) still depend greatly on the use of film radiographs. Additionally, there is a scarcity of specialty-trained or formally licensed radiologists in many of these regions. Subsequently, there is great potential for the use of smartphone capture of plain radiograph films which would allow for increased access to economical and efficient consultation from board-certified radiologists abroad. The present study addresses the ability to diagnose a subset of radiographic findings identified on both the original film radiograph and the captured camera phone image.

Legal, ethical and practical considerations of smartphone use in orthopaedics

How many of us are in breach of basic governance requirements?

Implementing a radiology-information technology project: mobile image viewing use case and a general guideline for radiologist-information technology team collaboration

OBJECTIVE

This article illustrates the importance of radiologist engagement in the successful implementation of radiology-information technology (IT) projects through the example of establishing a mobile image viewing solution for health care professionals.

CONCLUSION

With an understanding of the types of decisions that benefit from radiologist input, this article outlines an overall project framework to provide a context for how radiologists might engage in the project cycle.

The posterior impingement view: an alternative conventional projection to detect bony posterior ankle impingement

PURPOSE

The purpose of the current study was to clinically evaluate the diagnostic value of the new posterior impingement (PIM) view in the detection of an os trigonum, compared with the standard lateral view, using computed tomography (CT) as a reference standard.

METHODS

Three observers, 2 experienced (orthopaedic surgeon and radiologist) and one inexperienced (resident), independently scored 142 radiographic images for the presence of an os trigonum. The diagnostic performance was assessed using the computed tomographic scan as the reference standard. Accuracy, sensitivity, specificity, positive predicted value (PPV), and negative predicted value (NPV) were calculated.

RESULTS

The PIM view had significantly superior accuracy compared with the lateral view for each observer: orthopaedic surgeon, PIM view = 90 versus lateral view = 75 (P = .013); radiologist, PIM view = 80 versus lateral view = 64 (P = .019); resident, PIM view = 90 versus lateral view = 79 (P = .039). The mean sensitivity and specificity of the lateral view for all observers was 50% and 81%, respectively. For the PIM view, this was 78% and 89%, respectively. The PPV was 50% for the lateral view and 70% for the PIM view. The NPV was 84% for the lateral view and 93% for the PIM view.

CONCLUSIONS

The PIM view has significantly superior diagnostic accuracy compared with the conventional lateral view in the detection of an os trigonum. In cases of symptomatic posterior ankle impingement, we advise that a PIM view be used instead of or in addition to the standard lateral view for detection of posterior talar pathologic conditions. LEVEL OF 

EVIDENCE

Level II, development of diagnostic criteria on basis of consecutive patients with universally applied reference gold standard.