Development and Evaluation of a Quality Assessment Tool for Laparoscopic Sleeve Gastrectomy Videos: A Review and Comparison of Academic and Online Video Resources

BACKGROUND

Video recording of surgical procedures is increasing in popularity. They are presented in various platforms, many of which are not peer-reviewed. Laparoscopic sleeve gastrectomy (LSG) videos are widely available; however, there is limited evidence supporting the use of reporting guidelines when uploading LSG videos to create a valuable educational video. We aimed to determine the variations and establish the quality of published LSG videos, in both peer-reviewed literature and on YouTube, using a newly designed checklist to improve the quality and enhance the transparency of video reporting.

METHODS

A quality assessment tool was designed by using existing research and society guidelines, such as the Bariatric Metabolic Surgery Standardization (BMSS). A systematic review using PRISMA guidelines was performed on MEDLINE and EMBASE databases to identify video case reports (academic videos) and a similar search was performed on the commercial YouTube platform (commercial videos) simultaneously. All videos displaying LSG were reviewed and scored using the quality assessment tool. Academic and commercial videos were subsequently compared and an evidence-based checklist was created.

RESULTS

A total of 93 LSG recordings including 26 academic and 67 commercial videos were reviewed. Mean score of the checklist was 5/11 and 4/11 for videos published in articles and YouTube, respectively. Academic videos had higher rates of describing instruments used, such as orogastric tube (P < 0.001) and stapler information (P = 0.04). Fifty-four percent of academic videos described short-term patient outcomes, while not reported in commercial videos (P < 0.001). Sleeve resection status was not universally reported.

CONCLUSIONS

Videos published in the academic literature are describing steps in greater detail with more emphasis on specific technical elements and patient outcomes and thus have a higher educational value. A new quality assessment tool has been proposed for video reporting guidelines to improve the reliability and value of published video research.

Quality assessment of higher resolution images and videos with remote testing

In many research fields, human-annotated data plays an important role as it is used to accomplish a multitude of tasks. One such example is in the field of multimedia quality assessment where subjective annotations can be used to train or evaluate quality prediction models. Lab-based tests could be one approach to get such quality annotations. They are usually performed in well-defined and controlled environments to ensure high reliability. However, this high reliability comes at a cost of higher time consumption and costs incurred. To mitigate this, crowd or online tests could be used. Usually, online tests cover a wider range of end devices, environmental conditions, or participants, which may have an impact on the ratings. To verify whether such online tests can be used for visual quality assessment, we designed three online tests. These online tests are based on previously conducted lab tests as this enables comparison of the results of both test paradigms. Our focus is on the quality assessment of high-resolution images and videos. The online tests use AVrate Voyager, which is a publicly accessible framework for online tests. To transform the lab tests into online tests, dedicated adaptations in the test methodologies are required. The considered modifications are, for example, a patch-based or centre cropping of the images and videos, or a randomly sub-sampling of the to-be-rated stimuli. Based on the analysis of the test results in terms of correlation and SOS analysis it is shown that online tests can be used as a reliable replacement for lab tests albeit with some limitations. These limitations relate to, e.g., lack of appropriate display devices, limitation of web technologies, and modern browsers considering support for different video codecs and formats.

A neural network based framework for effective laparoscopic video quality assessment

Video quality assessment is a challenging problem having a critical significance in the context of medical imaging. For instance, in laparoscopic surgery, the acquired video data suffers from different kinds of distortion that not only hinder surgery performance but also affect the execution of subsequent tasks in surgical navigation and robotic surgeries. For this reason, we propose in this paper neural network-based approaches for distortion classification as well as quality prediction. More precisely, a Residual Network (ResNet) based approach is firstly developed for simultaneous ranking and classification task. Then, this architecture is extended to make it appropriate for the quality prediction task by using an additional Fully Connected Neural Network (FCNN). To train the overall architecture (ResNet and FCNN models), transfer learning and end-to-end learning approaches are investigated. Experimental results, carried out on a new laparoscopic video quality database, have shown the efficiency of the proposed methods compared to recent conventional and deep learning based approaches.

Development of the User Experience (UX) and Video Quality Evaluation (VQE) Instruments for Assessment of Intraoperative Video Capture Technology

OBJECTIVE

There exists a lack of technology to reliably and routinely capture high-quality video of open surgical procedures. To critically evaluate and compare new and existing technology solutions, we must have widely accepted evaluation criteria for intraoperative camera devices. The objective, therefore, was to develop evaluation criteria for intraoperative camera devices, as well as the video product they produce.

DESIGN

A modified Delphi process that included 2 iterative surveys was used to build expert consensus and develop 2 evaluation instruments: one to evaluate the user experience (UX) of using an intraoperative camera device, and the second for video quality evaluation (VQE) of the video product.

SETTING

Global, through iterative online surveys.

PARTICIPANTS

Surgeons who perform open surgery and have experience with intraoperative video capture.

RESULTS

Eighty-six experts participated in the first iteration of the survey and 46 in the second. Ten factors met the a priori cutoff for >80% agreement for the UX survey: (1) ease of setup/integration with current practice, (2) comfort, (3) distracting during case, (4) overall satisfaction with wearing the device, (5) would you use this device again, (6) would you recommend this device to colleagues, (7) the weight of wearing the device, (8) sufficient battery life, (9) ability to control device while operating, and (10) degree to which the device interferes or is incompatible with other surgical accessories. Six factors met the cutoff for the VQE survey: (1) camera stability, (2) brightness/exposure, (3) resolution/sharpness, (4) unobstructed view of the surgical field, (5) appropriate field of view, and (6) overall satisfaction with video quality.

CONCLUSIONS

These instruments can be used to critically evaluate camera technologies for intraoperative video capture of open surgery.